2024-04-26T21:32:16.362 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:32:16.362 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230 --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/orig.config.yaml 2024-04-26T21:32:16.408 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:32:16.498 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230 branch: wip-rishabh-testing-20240426.111959 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7675230' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGvZg1Zby3QB9t81i0ElWC414PbVAsNiey83rUHUZCko6B8eS6pLsKEIMacb1QCNwzf/k21snr7aND/AUTJfSgk= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvsDwOjxTbut750N71/3tg1jCt0r43pR8LS3tiL6W+phHCPmd1t7l2VKVcjeP7Oe+yPoy4+oW+C+So7jUXfp1M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:32:16.498 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:32:16.499 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:32:16.499 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:32:16.501 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:32:16.524 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:32:16.525 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:32:16.525 INFO:teuthology.packaging:ref: None 2024-04-26T21:32:16.525 INFO:teuthology.packaging:tag: None 2024-04-26T21:32:16.525 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:32:16.525 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:32:16.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:32:16.748 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:32:16.750 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:32:16.756 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:32:16.756 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:32:16.797 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:32:16.814 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:32:16.838 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:32:16.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:27:04.591834', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGvZg1Zby3QB9t81i0ElWC414PbVAsNiey83rUHUZCko6B8eS6pLsKEIMacb1QCNwzf/k21snr7aND/AUTJfSgk='} 2024-04-26T21:32:16.882 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:27:04.590358', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvsDwOjxTbut750N71/3tg1jCt0r43pR8LS3tiL6W+phHCPmd1t7l2VKVcjeP7Oe+yPoy4+oW+C+So7jUXfp1M='} 2024-04-26T21:32:16.883 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:32:16.889 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T21:32:16.889 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T21:32:16.889 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:32:16.977 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f101cbb3160>, signals=[15]) 2024-04-26T21:32:16.978 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:32:16.983 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:32:16.983 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:32:16.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:17.061 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:32:17.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:17.133 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:32:17.140 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-26T21:32:17.157 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-26T21:32:17.157 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-26T21:32:17.214 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:32:17.215 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:32:17.215 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:32:17.215 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:32:17.215 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:32:17.215 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:32:17.215 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-26T21:32:17.237 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-04-26T21:32:17.254 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-04-26T21:32:17.254 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-04-26T21:32:17.310 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-04-26T21:32:17.310 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:32:17.311 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:32:17.312 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-04-26T21:32:17.337 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:32:17.347 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:32:17.391 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:32:17.391 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:32:17.393 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:32:17.409 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:32:17.447 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:32:17.447 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:32:17.449 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:32:17.484 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:32:17.538 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:32:17.554 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:32:17.557 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T21:32:17.558 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:32:17.558 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:32:17.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:32:17.575 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.575 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:32:17.575 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-26T21:32:17.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:32:17.577 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.577 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:32:17.577 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-26T21:32:17.632 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:32:17.632 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.633 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:32:17.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.637 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:32:17.638 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.638 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:32:17.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:17.781 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum upgrade -y linux-firmware 2024-04-26T21:32:17.852 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum upgrade -y linux-firmware 2024-04-26T21:32:20.115 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T21:32:20.553 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T21:32:21.917 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-26T21:32:22.313 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-26T21:32:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-26T21:32:26.247 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-26T21:32:27.827 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T21:32:27.881 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-26T21:32:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-26T21:32:29.277 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T21:32:34.170 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T21:32:34.249 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T21:32:36.105 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-04-26T21:32:36.106 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-04-26T21:32:36.106 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-04-26T21:32:36.202 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:32:36.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:36.214 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-26T21:32:36.215 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do. 2024-04-26T21:32:36.215 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-26T21:32:36.334 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:32:36.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:32:36.427 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:32:36.427 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:32:36.430 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-04-26T21:32:36.550 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:32:36.551 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:32:36.555 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi106.front.sepia.ceph.com to complete... 2024-04-26T21:33:25.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26 21:33:25 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:33:25.206 INFO:teuthology.orchestra.run.smithi106.stderr:FINISHED --2024-04-26 21:33:25-- 2024-04-26T21:33:25.206 INFO:teuthology.orchestra.run.smithi106.stderr:Total wall clock time: 49s 2024-04-26T21:33:25.206 INFO:teuthology.orchestra.run.smithi106.stderr:Downloaded: 1 files, 341M in 48s (7.10 MB/s) 2024-04-26T21:33:25.211 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:33:25.211 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:33:27.341 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying... ######################################## 2024-04-26T21:33:27.495 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing... ######################################## 2024-04-26T21:33:27.599 INFO:teuthology.orchestra.run.smithi106.stdout:Updating / installing... 2024-04-26T21:33:35.523 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:33:37.984 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26 21:33:37 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:33:37.984 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-04-26 21:33:37-- 2024-04-26T21:33:37.984 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 1m 2s 2024-04-26T21:33:37.984 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 341M in 1m 1s (5.58 MB/s) 2024-04-26T21:33:37.987 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:33:37.987 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:33:40.342 INFO:teuthology.orchestra.run.smithi204.stdout:Verifying... ######################################## 2024-04-26T21:33:40.569 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing... ######################################## 2024-04-26T21:33:40.677 INFO:teuthology.orchestra.run.smithi204.stdout:Updating / installing... 2024-04-26T21:33:48.704 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:34:13.117 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:34:13.118 DEBUG:teuthology.orchestra.run.smithi106:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:34:13.211 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:34:13.212 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:34:13.213 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:34:13.214 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:34:13.215 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:34:13.216 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:34:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:34:13.218 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:34:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:34:13.220 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:34:13.221 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:34:13.222 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:34:13.223 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:34:13.224 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:34:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:34:13.226 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:34:13.227 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:34:13.228 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:34:13.229 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:34:13.230 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:34:13.231 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:34:13.232 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:34:13.233 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:34:13.234 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:34:13.235 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:34:13.236 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:34:13.237 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:34:13.238 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:34:13.239 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:34:13.240 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:34:13.241 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:34:13.242 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:34:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:34:13.244 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:34:13.245 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:34:13.246 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:34:13.247 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:34:13.248 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:34:13.249 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:34:13.251 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:34:13.252 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:34:13.253 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:34:13.254 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:34:13.255 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:34:13.256 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:34:13.257 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:34:13.258 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:34:13.259 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:34:13.260 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:34:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:34:13.262 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:34:13.263 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:34:13.264 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:34:13.265 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:34:13.266 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:34:13.267 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:34:13.268 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:34:13.269 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:34:13.270 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:34:13.271 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:34:13.272 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:34:13.273 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:34:13.274 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:34:13.275 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:34:13.276 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:34:13.277 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:34:13.278 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:34:13.279 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:34:13.280 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:34:13.281 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:34:13.282 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:34:13.283 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:34:13.284 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:34:13.285 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:34:13.286 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:34:13.287 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:34:13.290 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:34:13.291 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:34:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:34:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:34:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:34:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:34:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:34:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:34:13.298 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:34:13.299 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:34:13.300 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:34:13.301 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:34:13.302 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:34:13.303 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:34:13.304 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:34:13.305 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:34:13.306 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:34:13.307 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:34:13.308 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:34:13.309 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:34:13.310 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:34:13.311 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:34:13.312 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:34:13.313 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:34:13.314 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:34:13.315 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:34:13.316 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:34:13.318 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:34:13.319 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:34:13.320 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:34:13.321 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:34:13.322 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:34:13.323 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:34:13.324 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:34:13.325 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:34:13.326 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:34:13.327 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:34:13.328 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:34:13.329 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:34:13.331 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:34:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:34:13.333 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:34:13.334 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:34:13.335 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:34:13.336 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:34:13.337 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:34:13.338 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:34:13.339 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:34:13.340 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:34:13.341 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:34:13.342 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:34:13.343 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:34:13.344 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:34:13.345 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:34:13.346 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:34:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:34:13.348 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:34:13.349 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:34:13.350 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:34:13.351 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:34:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:34:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:34:13.354 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:34:13.355 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:34:13.356 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:34:13.357 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:34:13.358 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:34:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:34:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:34:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:34:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:34:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:34:13.363 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.363 DEBUG:teuthology.orchestra.run.smithi106:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:34:13.386 INFO:teuthology.orchestra.run.smithi106.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout: fi 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:done 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.387 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-26T21:34:13.388 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2-tools 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2-tools 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.06 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 68.el9 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2024-04-26T21:34:13.461 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Group : Unspecified 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 8264254 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : builder@centos.org 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Support tools for GRUB. 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:34:13.462 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:34:13.463 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:34:13.463 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:34:13.463 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2024-04-26T21:34:13.463 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:34:13.463 INFO:teuthology.orchestra.run.smithi106.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:34:13.464 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:34:13.464 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:13.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:34:13.998 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2024-04-26T21:34:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:34:15.429 INFO:teuthology.orchestra.run.smithi106.stderr:done 2024-04-26T21:34:15.431 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-26T21:34:15.466 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.QkqSdrTW5j 2024-04-26T21:34:15.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QkqSdrTW5j 2024-04-26T21:34:15.541 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.QkqSdrTW5j 2024-04-26T21:34:15.686 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.QkqSdrTW5j is 6KB 2024-04-26T21:34:15.735 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.QkqSdrTW5j 2024-04-26T21:34:15.749 DEBUG:teuthology.orchestra.run.smithi106:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:34:15.810 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:34:15.810 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:34:15.811 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:34:15.811 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:34:15.812 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:15.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2024-04-26T21:34:25.977 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:34:25.977 DEBUG:teuthology.orchestra.run.smithi204:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:34:26.058 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.058 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.058 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.058 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:34:26.058 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:34:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:34:26.060 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:34:26.061 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:34:26.062 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:34:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:34:26.064 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:34:26.065 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:34:26.066 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:34:26.067 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:34:26.068 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:34:26.069 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:34:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:34:26.071 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:34:26.072 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:34:26.073 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:34:26.074 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:34:26.075 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:34:26.076 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:34:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:34:26.078 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:34:26.079 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:34:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:34:26.081 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:34:26.082 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:34:26.083 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:34:26.084 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:34:26.085 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:34:26.086 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:34:26.087 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:34:26.088 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:34:26.089 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:34:26.090 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:34:26.091 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:34:26.092 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:34:26.093 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:34:26.094 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:34:26.095 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:34:26.096 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:34:26.097 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:34:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:34:26.099 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:34:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:34:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:34:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:34:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:34:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:34:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:34:26.106 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:34:26.107 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:34:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:34:26.109 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:34:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:34:26.111 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:34:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:34:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:34:26.114 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:34:26.115 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:34:26.116 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:34:26.117 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:34:26.118 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:34:26.119 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:34:26.120 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:34:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:34:26.122 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:34:26.123 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:34:26.124 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:34:26.125 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:34:26.126 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:34:26.127 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:34:26.128 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:34:26.129 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:34:26.130 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:34:26.131 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:34:26.132 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:34:26.133 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:34:26.134 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:34:26.135 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:34:26.136 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:34:26.137 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:34:26.138 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:34:26.140 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:34:26.141 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:34:26.142 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:34:26.143 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:34:26.144 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:34:26.145 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:34:26.146 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:34:26.147 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:34:26.148 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:34:26.149 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:34:26.150 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:34:26.151 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:34:26.152 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:34:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:34:26.154 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:34:26.155 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:34:26.156 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:34:26.157 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:34:26.158 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:34:26.159 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:34:26.160 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:34:26.161 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:34:26.162 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:34:26.163 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:34:26.164 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:34:26.165 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:34:26.166 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:34:26.167 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:34:26.168 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:34:26.169 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:34:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:34:26.171 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:34:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:34:26.173 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:34:26.174 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:34:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:34:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:34:26.177 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:34:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:34:26.179 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:34:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:34:26.181 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:34:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:34:26.183 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:34:26.184 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:34:26.185 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:34:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:34:26.187 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:34:26.188 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:34:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:34:26.190 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:34:26.191 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:34:26.192 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:34:26.193 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:34:26.194 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:34:26.195 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:34:26.196 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:34:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:34:26.198 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:34:26.199 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:34:26.200 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:34:26.201 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:34:26.202 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:34:26.203 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:34:26.204 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:34:26.205 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:34:26.209 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.209 DEBUG:teuthology.orchestra.run.smithi204:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:34:26.231 INFO:teuthology.orchestra.run.smithi204.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:34:26.231 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: fi 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:34:26.232 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.233 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T21:34:26.234 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-04-26T21:34:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:34:26.307 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:34:26.308 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-04-26T21:34:26.308 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:34:26.308 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:34:26.309 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:34:26.309 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:26.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:34:26.861 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-04-26T21:34:28.274 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:34:28.303 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-04-26T21:34:28.305 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-26T21:34:28.320 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.BOmdLwHbT9 2024-04-26T21:34:28.320 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BOmdLwHbT9 2024-04-26T21:34:28.386 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.BOmdLwHbT9 2024-04-26T21:34:28.526 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.BOmdLwHbT9 is 6KB 2024-04-26T21:34:28.574 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.BOmdLwHbT9 2024-04-26T21:34:28.588 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:34:28.659 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:34:28.659 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:34:28.659 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:34:28.660 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:34:28.661 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:34:28.802 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-04-26T21:34:45.991 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T21:34:45.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:34:45.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:34:58.814 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T21:34:58.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:34:58.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:04.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-04-26T21:35:13.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:35:13.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:17.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-26T21:35:26.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:35:26.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:29.346 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-26T21:35:41.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:35:41.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:59.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-26T21:36:13.502 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T21:36:14.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:36:14.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:36:17.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-26T21:36:25.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:36:25.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:36:25.912 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-26T21:36:26.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:36:26.520 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:36:26.521 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-26T21:36:26.574 INFO:teuthology.orchestra.run.smithi106.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:36:26.574 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:36:26.574 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:36:26.574 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:36:27.575 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T21:36:27.575 DEBUG:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:36:27.708 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2024-04-26T21:36:27.770 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:35.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:36:35.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:36:36.162 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-26T21:36:36.755 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:36:36.755 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:36:36.756 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-26T21:36:36.807 INFO:teuthology.orchestra.run.smithi204.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:36:36.807 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:36:36.807 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:36:36.807 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:36:37.808 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T21:36:37.808 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:36:37.988 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-04-26T21:36:38.144 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:38.145 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:36:38.155 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:36:38.155 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:36:38.163 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:36:38.212 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:36:38.219 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:36:38.240 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:36:38.240 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:36:38.246 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:36:38.362 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:36:38.369 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:36:38.369 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:36:38.376 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:36:38.461 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:36:38.461 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:36:38.490 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:36:38.495 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:36:38.499 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:36:38.507 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:36:38.507 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:36:38.512 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:36:38.592 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:36:38.603 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:36:38.603 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:36:38.612 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:36:38.662 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:38.738 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:38.826 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:38.904 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:38.988 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:36:38.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:36:39.071 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:39.156 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:39.250 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:39.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:39.393 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:36:39.394 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:36:39.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-26T21:36:39.470 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-04-26T21:36:39.587 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:36:39.617 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:36:39.899 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:36:39.907 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:36:39.907 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:36:39.944 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:36:39.976 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:36:39.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-26T21:36:40.117 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-26T21:36:40.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-04-26T21:36:40.244 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-04-26T21:36:40.249 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:36:40.249 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-26T21:36:40.321 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-26T21:36:40.321 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-04-26T21:36:40.381 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-04-26T21:36:40.381 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:36:40.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:36:40.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:36:40.451 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:36:40.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:36:40.537 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:36:40.538 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:36:40.608 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:36:40.691 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:36:40.700 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:36:40.720 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:36:40.721 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:41:46.436 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-04-26T21:41:46.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:41:46.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:41:46.546 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-26T21:41:46.684 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-26T21:41:46.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:41:46.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:41:46.785 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-26T21:41:46.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T21:41:46.944 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:41:46.955 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:41:46.955 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:41:46.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:41:46.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:41:46.964 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:41:47.047 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:41:47.052 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:41:47.086 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:41:47.096 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:41:47.162 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-04-26T21:41:47.169 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-26T21:41:47.195 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-04-26T21:41:47.199 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-26T21:41:47.236 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:41:47.238 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:41:47.277 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:41:47.277 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:41:47.405 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:41:47.406 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:41:47.414 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:41:47.414 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-26T21:41:47.414 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.414 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.414 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.415 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:41:47.417 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:41:47.424 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:41:47.425 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:41:47.425 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:41:47.425 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:41:47.430 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:41:47.430 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:41:47.431 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:41:47.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:47.433 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:41:47.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:47.756 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:41:47.756 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:41:47.761 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:41:47.761 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:41:48.011 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:41:48.011 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:41:48.011 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:41:48.099 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:41:48.100 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:41:48.154 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:41:48.154 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:41:48.154 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:41:48.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:41:48.238 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:41:48.238 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:41:48.341 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:41:48.366 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:41:48.421 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-04-26T21:41:48.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-04-26T21:41:48.478 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:41:48.556 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-26T21:41:48.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-26T21:41:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:48 files removed 2024-04-26T21:41:48.788 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:41:48.872 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-26T21:41:48.911 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:41:49.949 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 115 kB/s | 84 kB 00:00 2024-04-26T21:41:50.003 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 125 kB/s | 84 kB 00:00 2024-04-26T21:41:50.729 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-04-26T21:41:51.064 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 11 kB/s | 12 kB 00:01 2024-04-26T21:41:51.330 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-26T21:41:51.666 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-26T21:41:52.736 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-04-26T21:41:53.290 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-04-26T21:41:56.696 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-04-26T21:41:58.107 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-04-26T21:42:01.138 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-26T21:42:03.016 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-26T21:42:03.359 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-04-26T21:42:04.867 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T21:42:07.075 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T21:42:07.105 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 21 MB 00:03 2024-04-26T21:42:12.946 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-04-26T21:42:12.999 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T21:42:15.386 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.388 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.409 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:42:15.410 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.411 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:42:15.413 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:42:15.415 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:42:15.418 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:42:15.420 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:42:15.440 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.442 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.462 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:42:15.463 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:42:15.465 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:42:15.466 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:42:15.469 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:42:15.472 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:42:15.473 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:42:15.490 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-04-26T21:42:15.506 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T21:42:15.506 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-04-26T21:42:15.506 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T21:42:15.506 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:42:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:42:15.508 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:42:15.509 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:42:15.510 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:42:15.511 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:42:15.512 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:42:15.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:42:15.514 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:42:15.515 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:42:15.516 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:42:15.517 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-04-26T21:42:15.518 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T21:42:15.519 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-04-26T21:42:15.519 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 4 Packages 2024-04-26T21:42:15.519 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:42:15.519 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 222 M 2024-04-26T21:42:15.519 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-04-26T21:42:15.543 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-26T21:42:15.566 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-26T21:42:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-26T21:42:15.566 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:42:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:42:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:42:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:42:15.570 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:42:15.571 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:42:15.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:42:15.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:42:15.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:42:15.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:42:15.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:42:15.577 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 4 Packages 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:42:15.578 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 222 M 2024-04-26T21:42:15.579 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-26T21:42:16.979 INFO:teuthology.orchestra.run.smithi204.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-26T21:42:17.059 INFO:teuthology.orchestra.run.smithi106.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-26T21:42:17.368 INFO:teuthology.orchestra.run.smithi106.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.8 MB/s | 889 kB 00:00 2024-04-26T21:42:17.471 INFO:teuthology.orchestra.run.smithi106.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T21:42:17.546 INFO:teuthology.orchestra.run.smithi204.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.5 MB/s | 889 kB 00:00 2024-04-26T21:42:17.662 INFO:teuthology.orchestra.run.smithi204.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.2 MB/s | 145 kB 00:00 2024-04-26T21:42:18.060 INFO:teuthology.orchestra.run.smithi106.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-04-26T21:42:18.463 INFO:teuthology.orchestra.run.smithi204.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-04-26T21:42:18.485 INFO:teuthology.orchestra.run.smithi106.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-04-26T21:42:18.738 INFO:teuthology.orchestra.run.smithi204.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-26T21:42:18.847 INFO:teuthology.orchestra.run.smithi204.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-26T21:42:20.344 INFO:teuthology.orchestra.run.smithi106.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-26T21:42:21.022 INFO:teuthology.orchestra.run.smithi204.stdout:(7/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 7.7 MB/s | 17 MB 00:02 2024-04-26T21:42:21.147 INFO:teuthology.orchestra.run.smithi204.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-26T21:42:21.272 INFO:teuthology.orchestra.run.smithi204.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 202 kB/s | 25 kB 00:00 2024-04-26T21:42:23.164 INFO:teuthology.orchestra.run.smithi204.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.9 MB/s | 11 MB 00:02 2024-04-26T21:42:23.281 INFO:teuthology.orchestra.run.smithi204.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi204.stdout:(12/152): ceph-common-19.0.0-3306.gbc612ffb.el9 3.0 MB/s | 21 MB 00:06 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi204.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 93 kB/s | 27 kB 00:00 2024-04-26T21:42:23.649 INFO:teuthology.orchestra.run.smithi204.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 320 kB/s | 32 kB 00:00 2024-04-26T21:42:23.849 INFO:teuthology.orchestra.run.smithi204.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 819 kB/s | 163 kB 00:00 2024-04-26T21:42:23.941 INFO:teuthology.orchestra.run.smithi204.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.0 MB/s | 747 kB 00:00 2024-04-26T21:42:23.966 INFO:teuthology.orchestra.run.smithi204.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.1 MB/s | 127 kB 00:00 2024-04-26T21:42:24.133 INFO:teuthology.orchestra.run.smithi204.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.6 MB/s | 501 kB 00:00 2024-04-26T21:42:24.234 INFO:teuthology.orchestra.run.smithi204.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 453 kB/s | 45 kB 00:00 2024-04-26T21:42:24.285 INFO:teuthology.orchestra.run.smithi106.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 733 kB/s | 5.4 MB 00:07 2024-04-26T21:42:24.342 INFO:teuthology.orchestra.run.smithi204.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:42:24.476 INFO:teuthology.orchestra.run.smithi204.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.2 MB/s | 161 kB 00:00 2024-04-26T21:42:24.602 INFO:teuthology.orchestra.run.smithi204.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.5 MB/s | 321 kB 00:00 2024-04-26T21:42:24.726 INFO:teuthology.orchestra.run.smithi204.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T21:42:24.843 INFO:teuthology.orchestra.run.smithi204.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 854 kB/s | 99 kB 00:00 2024-04-26T21:42:24.952 INFO:teuthology.orchestra.run.smithi204.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 791 kB/s | 85 kB 00:00 2024-04-26T21:42:25.911 INFO:teuthology.orchestra.run.smithi204.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 2.8 MB/s | 5.4 MB 00:01 2024-04-26T21:42:25.986 INFO:teuthology.orchestra.run.smithi204.stdout:(27/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T21:42:26.020 INFO:teuthology.orchestra.run.smithi204.stdout:(28/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-26T21:42:26.112 INFO:teuthology.orchestra.run.smithi204.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 276 kB/s | 34 kB 00:00 2024-04-26T21:42:26.170 INFO:teuthology.orchestra.run.smithi204.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 948 kB/s | 142 kB 00:00 2024-04-26T21:42:26.594 INFO:teuthology.orchestra.run.smithi106.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 2.7 MB/s | 17 MB 00:06 2024-04-26T21:42:26.702 INFO:teuthology.orchestra.run.smithi106.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 235 kB/s | 25 kB 00:00 2024-04-26T21:42:27.096 INFO:teuthology.orchestra.run.smithi204.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 3.6 MB/s | 3.5 MB 00:00 2024-04-26T21:42:27.212 INFO:teuthology.orchestra.run.smithi204.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.1 MB/s | 249 kB 00:00 2024-04-26T21:42:27.296 INFO:teuthology.orchestra.run.smithi204.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 597 kB/s | 49 kB 00:00 2024-04-26T21:42:27.405 INFO:teuthology.orchestra.run.smithi204.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 153 kB/s | 16 kB 00:00 2024-04-26T21:42:27.513 INFO:teuthology.orchestra.run.smithi204.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 273 kB 00:00 2024-04-26T21:42:27.906 INFO:teuthology.orchestra.run.smithi204.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-26T21:42:27.998 INFO:teuthology.orchestra.run.smithi204.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 202 kB/s | 18 kB 00:00 2024-04-26T21:42:28.514 INFO:teuthology.orchestra.run.smithi204.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-04-26T21:42:28.815 INFO:teuthology.orchestra.run.smithi204.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 98 kB/s | 80 kB 00:00 2024-04-26T21:42:29.690 INFO:teuthology.orchestra.run.smithi204.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 223 kB/s | 195 kB 00:00 2024-04-26T21:42:30.182 INFO:teuthology.orchestra.run.smithi204.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 482 kB/s | 803 kB 00:01 2024-04-26T21:42:30.591 INFO:teuthology.orchestra.run.smithi204.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 282 kB/s | 253 kB 00:00 2024-04-26T21:42:31.299 INFO:teuthology.orchestra.run.smithi204.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 151 kB/s | 106 kB 00:00 2024-04-26T21:42:32.000 INFO:teuthology.orchestra.run.smithi204.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 193 kB/s | 135 kB 00:00 2024-04-26T21:42:32.150 INFO:teuthology.orchestra.run.smithi204.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 639 kB/s | 1.2 MB 00:01 2024-04-26T21:42:32.825 INFO:teuthology.orchestra.run.smithi204.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 152 kB/s | 125 kB 00:00 2024-04-26T21:42:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 1.2 MB/s | 11 MB 00:08 2024-04-26T21:42:33.025 INFO:teuthology.orchestra.run.smithi204.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 245 kB/s | 215 kB 00:00 2024-04-26T21:42:33.059 INFO:teuthology.orchestra.run.smithi204.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-04-26T21:42:33.060 INFO:teuthology.orchestra.run.smithi106.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.3 MB/s | 225 kB 00:00 2024-04-26T21:42:33.109 INFO:teuthology.orchestra.run.smithi204.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T21:42:33.176 INFO:teuthology.orchestra.run.smithi204.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 326 kB/s | 21 kB 00:00 2024-04-26T21:42:33.177 INFO:teuthology.orchestra.run.smithi106.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 233 kB/s | 27 kB 00:00 2024-04-26T21:42:33.243 INFO:teuthology.orchestra.run.smithi204.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.7 MB/s | 313 kB 00:00 2024-04-26T21:42:33.285 INFO:teuthology.orchestra.run.smithi204.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-04-26T21:42:33.327 INFO:teuthology.orchestra.run.smithi204.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 395 kB/s | 33 kB 00:00 2024-04-26T21:42:33.328 INFO:teuthology.orchestra.run.smithi106.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 212 kB/s | 32 kB 00:00 2024-04-26T21:42:33.454 INFO:teuthology.orchestra.run.smithi204.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 146 kB/s | 18 kB 00:00 2024-04-26T21:42:33.528 INFO:teuthology.orchestra.run.smithi204.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 152 kB/s | 11 kB 00:00 2024-04-26T21:42:33.603 INFO:teuthology.orchestra.run.smithi204.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-04-26T21:42:33.653 INFO:teuthology.orchestra.run.smithi204.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T21:42:33.695 INFO:teuthology.orchestra.run.smithi204.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 496 kB/s | 45 kB 00:00 2024-04-26T21:42:33.737 INFO:teuthology.orchestra.run.smithi204.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-26T21:42:33.788 INFO:teuthology.orchestra.run.smithi204.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-26T21:42:33.820 INFO:teuthology.orchestra.run.smithi106.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.5 MB/s | 747 kB 00:00 2024-04-26T21:42:33.830 INFO:teuthology.orchestra.run.smithi204.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 279 kB/s | 25 kB 00:00 2024-04-26T21:42:33.872 INFO:teuthology.orchestra.run.smithi204.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 573 kB/s | 47 kB 00:00 2024-04-26T21:42:33.905 INFO:teuthology.orchestra.run.smithi204.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-26T21:42:33.938 INFO:teuthology.orchestra.run.smithi204.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-26T21:42:33.981 INFO:teuthology.orchestra.run.smithi204.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 471 kB/s | 35 kB 00:00 2024-04-26T21:42:34.003 INFO:teuthology.orchestra.run.smithi106.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 896 kB/s | 163 kB 00:00 2024-04-26T21:42:34.128 INFO:teuthology.orchestra.run.smithi106.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.0 MB/s | 127 kB 00:00 2024-04-26T21:42:34.215 INFO:teuthology.orchestra.run.smithi204.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-26T21:42:34.323 INFO:teuthology.orchestra.run.smithi204.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-26T21:42:34.390 INFO:teuthology.orchestra.run.smithi204.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-26T21:42:34.474 INFO:teuthology.orchestra.run.smithi204.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-04-26T21:42:34.507 INFO:teuthology.orchestra.run.smithi204.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-26T21:42:34.612 INFO:teuthology.orchestra.run.smithi106.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 1.0 MB/s | 501 kB 00:00 2024-04-26T21:42:34.616 INFO:teuthology.orchestra.run.smithi204.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-26T21:42:34.683 INFO:teuthology.orchestra.run.smithi204.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T21:42:34.833 INFO:teuthology.orchestra.run.smithi204.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-26T21:42:34.926 INFO:teuthology.orchestra.run.smithi204.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T21:42:34.984 INFO:teuthology.orchestra.run.smithi204.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-26T21:42:35.110 INFO:teuthology.orchestra.run.smithi204.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-26T21:42:35.135 INFO:teuthology.orchestra.run.smithi204.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 171 kB/s | 26 kB 00:00 2024-04-26T21:42:35.160 INFO:teuthology.orchestra.run.smithi204.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 389 kB/s | 19 kB 00:00 2024-04-26T21:42:35.194 INFO:teuthology.orchestra.run.smithi204.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:42:35.219 INFO:teuthology.orchestra.run.smithi204.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-26T21:42:35.286 INFO:teuthology.orchestra.run.smithi204.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-26T21:42:35.361 INFO:teuthology.orchestra.run.smithi204.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-26T21:42:35.470 INFO:teuthology.orchestra.run.smithi204.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-26T21:42:35.495 INFO:teuthology.orchestra.run.smithi204.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 580 kB/s | 77 kB 00:00 2024-04-26T21:42:35.545 INFO:teuthology.orchestra.run.smithi204.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 866 kB/s | 42 kB 00:00 2024-04-26T21:42:35.579 INFO:teuthology.orchestra.run.smithi204.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-26T21:42:35.612 INFO:teuthology.orchestra.run.smithi204.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-26T21:42:35.646 INFO:teuthology.orchestra.run.smithi204.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T21:42:35.673 INFO:teuthology.orchestra.run.smithi204.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 879 kB/s | 51 kB 00:00 2024-04-26T21:42:35.697 INFO:teuthology.orchestra.run.smithi204.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-26T21:42:35.764 INFO:teuthology.orchestra.run.smithi204.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 632 kB/s | 42 kB 00:00 2024-04-26T21:42:35.814 INFO:teuthology.orchestra.run.smithi204.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 899 kB/s | 44 kB 00:00 2024-04-26T21:42:35.881 INFO:teuthology.orchestra.run.smithi204.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-26T21:42:35.939 INFO:teuthology.orchestra.run.smithi204.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 624 kB/s | 36 kB 00:00 2024-04-26T21:42:36.006 INFO:teuthology.orchestra.run.smithi204.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 720 kB/s | 47 kB 00:00 2024-04-26T21:42:36.082 INFO:teuthology.orchestra.run.smithi204.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-26T21:42:36.154 INFO:teuthology.orchestra.run.smithi204.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 816 kB/s | 58 kB 00:00 2024-04-26T21:42:36.279 INFO:teuthology.orchestra.run.smithi106.stdout:(18/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 21 MB 00:19 2024-04-26T21:42:36.363 INFO:teuthology.orchestra.run.smithi106.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 547 kB/s | 45 kB 00:00 2024-04-26T21:42:36.374 INFO:teuthology.orchestra.run.smithi204.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-26T21:42:36.400 INFO:teuthology.orchestra.run.smithi204.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 383 kB/s | 94 kB 00:00 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi204.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T21:42:36.453 INFO:teuthology.orchestra.run.smithi204.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 869 kB/s | 45 kB 00:00 2024-04-26T21:42:36.478 INFO:teuthology.orchestra.run.smithi204.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 433 kB/s | 22 kB 00:00 2024-04-26T21:42:36.555 INFO:teuthology.orchestra.run.smithi106.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 707 kB/s | 135 kB 00:00 2024-04-26T21:42:36.573 INFO:teuthology.orchestra.run.smithi204.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.1 MB/s | 863 kB 00:00 2024-04-26T21:42:36.680 INFO:teuthology.orchestra.run.smithi106.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.3 MB/s | 161 kB 00:00 2024-04-26T21:42:36.838 INFO:teuthology.orchestra.run.smithi106.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.0 MB/s | 321 kB 00:00 2024-04-26T21:42:37.022 INFO:teuthology.orchestra.run.smithi106.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.6 MB/s | 298 kB 00:00 2024-04-26T21:42:37.103 INFO:teuthology.orchestra.run.smithi204.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 878 kB/s | 548 kB 00:00 2024-04-26T21:42:37.122 INFO:teuthology.orchestra.run.smithi106.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.0 MB/s | 99 kB 00:00 2024-04-26T21:42:37.137 INFO:teuthology.orchestra.run.smithi204.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 549 kB/s | 309 kB 00:00 2024-04-26T21:42:37.204 INFO:teuthology.orchestra.run.smithi204.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-26T21:42:37.214 INFO:teuthology.orchestra.run.smithi106.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 939 kB/s | 85 kB 00:00 2024-04-26T21:42:37.321 INFO:teuthology.orchestra.run.smithi204.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-26T21:42:37.438 INFO:teuthology.orchestra.run.smithi204.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-04-26T21:42:37.563 INFO:teuthology.orchestra.run.smithi204.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 542 kB/s | 67 kB 00:00 2024-04-26T21:42:37.640 INFO:teuthology.orchestra.run.smithi106.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 5.4 MB 00:03 2024-04-26T21:42:37.672 INFO:teuthology.orchestra.run.smithi204.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-04-26T21:42:37.705 INFO:teuthology.orchestra.run.smithi204.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T21:42:37.743 INFO:teuthology.orchestra.run.smithi106.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 171 kB 00:00 2024-04-26T21:42:37.835 INFO:teuthology.orchestra.run.smithi204.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 839 kB 00:00 2024-04-26T21:42:37.840 INFO:teuthology.orchestra.run.smithi106.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 356 kB/s | 34 kB 00:00 2024-04-26T21:42:37.889 INFO:teuthology.orchestra.run.smithi204.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-26T21:42:37.915 INFO:teuthology.orchestra.run.smithi106.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-26T21:42:37.931 INFO:teuthology.orchestra.run.smithi204.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 457 kB/s | 43 kB 00:00 2024-04-26T21:42:37.990 INFO:teuthology.orchestra.run.smithi204.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-04-26T21:42:38.032 INFO:teuthology.orchestra.run.smithi204.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-26T21:42:38.115 INFO:teuthology.orchestra.run.smithi204.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-26T21:42:38.166 INFO:teuthology.orchestra.run.smithi204.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-26T21:42:38.232 INFO:teuthology.orchestra.run.smithi106.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T21:42:38.274 INFO:teuthology.orchestra.run.smithi204.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-26T21:42:38.300 INFO:teuthology.orchestra.run.smithi204.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 266 kB/s | 35 kB 00:00 2024-04-26T21:42:38.416 INFO:teuthology.orchestra.run.smithi204.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-04-26T21:42:38.636 INFO:teuthology.orchestra.run.smithi204.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi204.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 315 kB/s | 144 kB 00:00 2024-04-26T21:42:38.926 INFO:teuthology.orchestra.run.smithi204.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 37 kB/s | 11 kB 00:00 2024-04-26T21:42:39.001 INFO:teuthology.orchestra.run.smithi204.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-04-26T21:42:39.051 INFO:teuthology.orchestra.run.smithi204.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 186 kB/s | 23 kB 00:00 2024-04-26T21:42:39.108 INFO:teuthology.orchestra.run.smithi106.stdout:(31/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 4.0 MB/s | 49 MB 00:12 2024-04-26T21:42:39.158 INFO:teuthology.orchestra.run.smithi204.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 125 kB/s | 19 kB 00:00 2024-04-26T21:42:39.191 INFO:teuthology.orchestra.run.smithi106.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.0 MB/s | 249 kB 00:00 2024-04-26T21:42:39.211 INFO:teuthology.orchestra.run.smithi204.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-04-26T21:42:39.267 INFO:teuthology.orchestra.run.smithi106.stdout:(33/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.6 MB/s | 3.5 MB 00:01 2024-04-26T21:42:39.317 INFO:teuthology.orchestra.run.smithi106.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 396 kB/s | 49 kB 00:00 2024-04-26T21:42:39.336 INFO:teuthology.orchestra.run.smithi204.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 372 kB/s | 46 kB 00:00 2024-04-26T21:42:39.386 INFO:teuthology.orchestra.run.smithi204.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-26T21:42:39.389 INFO:teuthology.orchestra.run.smithi106.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 136 kB/s | 16 kB 00:00 2024-04-26T21:42:39.434 INFO:teuthology.orchestra.run.smithi106.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 273 kB 00:00 2024-04-26T21:42:39.437 INFO:teuthology.orchestra.run.smithi204.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 794 kB/s | 79 kB 00:00 2024-04-26T21:42:39.509 INFO:teuthology.orchestra.run.smithi106.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 245 kB/s | 18 kB 00:00 2024-04-26T21:42:39.529 INFO:teuthology.orchestra.run.smithi204.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 410 kB/s | 58 kB 00:00 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi204.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-26T21:42:39.618 INFO:teuthology.orchestra.run.smithi106.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.3 MB/s | 762 kB 00:00 2024-04-26T21:42:39.654 INFO:teuthology.orchestra.run.smithi204.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2024-04-26T21:42:39.688 INFO:teuthology.orchestra.run.smithi204.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 903 kB/s | 90 kB 00:00 2024-04-26T21:42:39.780 INFO:teuthology.orchestra.run.smithi204.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 227 kB/s | 28 kB 00:00 2024-04-26T21:42:39.830 INFO:teuthology.orchestra.run.smithi204.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-04-26T21:42:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-04-26T21:42:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-04-26T21:42:40.023 INFO:teuthology.orchestra.run.smithi204.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 253 kB/s | 36 kB 00:00 2024-04-26T21:42:40.082 INFO:teuthology.orchestra.run.smithi204.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 410 kB/s | 51 kB 00:00 2024-04-26T21:42:40.132 INFO:teuthology.orchestra.run.smithi204.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T21:42:40.143 INFO:teuthology.orchestra.run.smithi106.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-04-26T21:42:40.207 INFO:teuthology.orchestra.run.smithi204.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 720 kB/s | 90 kB 00:00 2024-04-26T21:42:40.257 INFO:teuthology.orchestra.run.smithi204.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-26T21:42:40.319 INFO:teuthology.orchestra.run.smithi106.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 99 kB/s | 80 kB 00:00 2024-04-26T21:42:40.333 INFO:teuthology.orchestra.run.smithi204.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-26T21:42:40.366 INFO:teuthology.orchestra.run.smithi204.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T21:42:40.483 INFO:teuthology.orchestra.run.smithi204.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T21:42:41.092 INFO:teuthology.orchestra.run.smithi204.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.7 MB/s | 3.4 MB 00:00 2024-04-26T21:42:41.232 INFO:teuthology.orchestra.run.smithi106.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 180 kB/s | 195 kB 00:01 2024-04-26T21:42:41.294 INFO:teuthology.orchestra.run.smithi106.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 260 kB/s | 253 kB 00:00 2024-04-26T21:42:41.317 INFO:teuthology.orchestra.run.smithi204.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-26T21:42:41.336 INFO:teuthology.orchestra.run.smithi106.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 468 kB/s | 803 kB 00:01 2024-04-26T21:42:42.061 INFO:teuthology.orchestra.run.smithi106.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 139 kB/s | 106 kB 00:00 2024-04-26T21:42:42.228 INFO:teuthology.orchestra.run.smithi106.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 152 kB/s | 135 kB 00:00 2024-04-26T21:42:43.028 INFO:teuthology.orchestra.run.smithi106.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 129 kB/s | 125 kB 00:00 2024-04-26T21:42:43.328 INFO:teuthology.orchestra.run.smithi106.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 195 kB/s | 215 kB 00:01 2024-04-26T21:42:43.567 INFO:teuthology.orchestra.run.smithi204.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 558 kB/s | 1.3 MB 00:02 2024-04-26T21:42:43.629 INFO:teuthology.orchestra.run.smithi106.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 524 kB/s | 1.2 MB 00:02 2024-04-26T21:42:44.562 INFO:teuthology.orchestra.run.smithi106.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 124 kB/s | 107 kB 00:00 2024-04-26T21:42:44.879 INFO:teuthology.orchestra.run.smithi106.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 373 kB/s | 689 kB 00:01 2024-04-26T21:42:45.201 INFO:teuthology.orchestra.run.smithi204.stdout:(151/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 2.1 MB/s | 49 MB 00:23 2024-04-26T21:42:45.371 INFO:teuthology.orchestra.run.smithi106.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 478 kB/s | 943 kB 00:01 2024-04-26T21:42:45.422 INFO:teuthology.orchestra.run.smithi106.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 25 kB/s | 21 kB 00:00 2024-04-26T21:42:45.576 INFO:teuthology.orchestra.run.smithi204.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 576 kB/s | 2.4 MB 00:04 2024-04-26T21:42:45.585 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:42:45.585 INFO:teuthology.orchestra.run.smithi204.stdout:Total 7.4 MB/s | 222 MB 00:30 2024-04-26T21:42:46.047 INFO:teuthology.orchestra.run.smithi106.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 269 kB/s | 313 kB 00:01 2024-04-26T21:42:46.097 INFO:teuthology.orchestra.run.smithi106.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 45 kB/s | 33 kB 00:00 2024-04-26T21:42:46.631 INFO:teuthology.orchestra.run.smithi106.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 18 kB 00:00 2024-04-26T21:42:46.673 INFO:teuthology.orchestra.run.smithi106.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-04-26T21:42:47.340 INFO:teuthology.orchestra.run.smithi106.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-04-26T21:42:47.440 INFO:teuthology.orchestra.run.smithi106.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 199 kB/s | 160 kB 00:00 2024-04-26T21:42:48.379 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-04-26T21:42:48.635 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-04-26T21:42:48.636 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-04-26T21:42:48.715 INFO:teuthology.orchestra.run.smithi106.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 457 kB/s | 582 kB 00:01 2024-04-26T21:42:48.757 INFO:teuthology.orchestra.run.smithi106.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 468 kB/s | 662 kB 00:01 2024-04-26T21:42:49.282 INFO:teuthology.orchestra.run.smithi106.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 45 kB/s | 25 kB 00:00 2024-04-26T21:42:49.383 INFO:teuthology.orchestra.run.smithi106.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-04-26T21:42:49.550 INFO:teuthology.orchestra.run.smithi106.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 742 kB/s | 3.0 MB 00:04 2024-04-26T21:42:50.266 INFO:teuthology.orchestra.run.smithi106.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-04-26T21:42:50.300 INFO:teuthology.orchestra.run.smithi106.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 206 kB/s | 188 kB 00:00 2024-04-26T21:42:50.392 INFO:teuthology.orchestra.run.smithi106.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 264 kB/s | 292 kB 00:01 2024-04-26T21:42:51.413 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-04-26T21:42:51.414 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-04-26T21:42:52.925 INFO:teuthology.orchestra.run.smithi106.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 642 kB/s | 1.6 MB 00:02 2024-04-26T21:42:54.809 INFO:teuthology.orchestra.run.smithi106.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 484 kB/s | 911 kB 00:01 2024-04-26T21:42:55.684 INFO:teuthology.orchestra.run.smithi106.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 153 kB/s | 133 kB 00:00 2024-04-26T21:42:56.501 INFO:teuthology.orchestra.run.smithi106.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 731 kB/s | 4.4 MB 00:06 2024-04-26T21:42:56.884 INFO:teuthology.orchestra.run.smithi106.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 734 kB/s | 4.7 MB 00:06 2024-04-26T21:42:57.643 INFO:teuthology.orchestra.run.smithi106.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 116 kB/s | 87 kB 00:00 2024-04-26T21:42:57.811 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-04-26T21:42:57.943 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:42:58.031 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:42:58.177 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:42:58.239 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:42:58.277 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:42:58.396 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:42:58.443 INFO:teuthology.orchestra.run.smithi106.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 532 kB/s | 1.0 MB 00:01 2024-04-26T21:42:58.831 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:42:58.949 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:42:59.126 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:42:59.167 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:42:59.296 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:42:59.349 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:42:59.393 INFO:teuthology.orchestra.run.smithi106.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 257 kB/s | 244 kB 00:00 2024-04-26T21:42:59.666 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:42:59.732 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:42:59.771 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:42:59.919 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:42:59.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:43:00.037 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:43:00.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:43:00.133 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:43:00.435 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:43:00.460 INFO:teuthology.orchestra.run.smithi106.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 233 kB/s | 248 kB 00:01 2024-04-26T21:43:00.499 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:43:00.703 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:43:00.829 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:43:00.874 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:43:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:43:01.026 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:43:01.078 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:43:01.118 INFO:teuthology.orchestra.run.smithi106.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 39 kB/s | 26 kB 00:00 2024-04-26T21:43:01.141 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:43:01.191 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:43:01.239 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:43:01.357 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:43:01.454 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:43:01.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:43:01.561 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:43:01.676 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:43:01.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:43:01.727 INFO:teuthology.orchestra.run.smithi106.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 32 kB/s | 19 kB 00:00 2024-04-26T21:43:01.765 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:43:01.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:43:01.945 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:43:02.006 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:43:02.113 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:43:02.298 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:43:02.367 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:43:02.536 INFO:teuthology.orchestra.run.smithi106.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 213 kB/s | 172 kB 00:00 2024-04-26T21:43:03.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:43:03.161 INFO:teuthology.orchestra.run.smithi106.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-26T21:43:03.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:43:03.242 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:43:03.288 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:43:03.329 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:43:03.391 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:43:03.663 INFO:teuthology.orchestra.run.smithi106.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 742 kB/s | 5.8 MB 00:07 2024-04-26T21:43:03.722 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:43:03.782 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:43:03.862 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:43:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:43:04.012 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:43:04.052 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:43:04.120 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:43:04.160 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:43:04.220 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:43:04.281 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:43:04.476 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:43:04.524 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:43:04.561 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:43:04.588 INFO:teuthology.orchestra.run.smithi106.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 271 kB/s | 250 kB 00:00 2024-04-26T21:43:04.610 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:43:04.737 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:43:04.805 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:43:04.860 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:43:04.894 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:43:04.927 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:43:05.128 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:43:05.338 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:43:05.375 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:43:05.416 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:43:05.469 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:43:05.528 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:43:05.580 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:43:05.581 INFO:teuthology.orchestra.run.smithi106.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 769 kB/s | 6.0 MB 00:07 2024-04-26T21:43:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 217 kB/s | 222 kB 00:01 2024-04-26T21:43:05.653 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:43:05.701 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:43:05.752 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:43:05.828 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:43:05.872 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:43:06.015 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:43:06.105 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:43:06.149 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:43:06.221 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:43:06.329 INFO:teuthology.orchestra.run.smithi106.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 104 kB/s | 77 kB 00:00 2024-04-26T21:43:06.365 INFO:teuthology.orchestra.run.smithi106.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 56 kB/s | 42 kB 00:00 2024-04-26T21:43:06.373 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:43:06.444 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:43:07.356 INFO:teuthology.orchestra.run.smithi106.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 160 kB/s | 159 kB 00:00 2024-04-26T21:43:07.407 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:43:07.440 INFO:teuthology.orchestra.run.smithi106.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 243 kB/s | 269 kB 00:01 2024-04-26T21:43:07.468 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:43:07.510 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:43:07.635 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:43:07.706 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:43:07.808 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:43:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:43:08.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:43:08.090 INFO:teuthology.orchestra.run.smithi106.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-04-26T21:43:08.148 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:43:08.284 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:43:08.315 INFO:teuthology.orchestra.run.smithi106.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 291 kB/s | 279 kB 00:00 2024-04-26T21:43:08.790 INFO:teuthology.orchestra.run.smithi106.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 77 kB/s | 54 kB 00:00 2024-04-26T21:43:09.328 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:43:09.432 INFO:teuthology.orchestra.run.smithi106.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-04-26T21:43:09.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:43:10.099 INFO:teuthology.orchestra.run.smithi106.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 752 kB/s | 5.1 MB 00:06 2024-04-26T21:43:10.150 INFO:teuthology.orchestra.run.smithi106.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 62 kB/s | 44 kB 00:00 2024-04-26T21:43:10.775 INFO:teuthology.orchestra.run.smithi106.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-04-26T21:43:10.983 INFO:teuthology.orchestra.run.smithi106.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 250 kB/s | 221 kB 00:00 2024-04-26T21:43:11.101 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:43:11.199 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:43:11.240 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:43:11.285 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:43:11.500 INFO:teuthology.orchestra.run.smithi106.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-04-26T21:43:11.663 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:43:11.704 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:43:11.824 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:43:11.860 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:43:11.927 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:43:11.984 INFO:teuthology.orchestra.run.smithi106.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 305 kB/s | 305 kB 00:00 2024-04-26T21:43:12.160 INFO:teuthology.orchestra.run.smithi106.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-04-26T21:43:12.478 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:43:12.515 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:43:12.615 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:43:12.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:43:12.760 INFO:teuthology.orchestra.run.smithi106.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 121 kB/s | 94 kB 00:00 2024-04-26T21:43:12.868 INFO:teuthology.orchestra.run.smithi106.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-04-26T21:43:13.144 INFO:teuthology.orchestra.run.smithi106.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 82 kB/s | 22 kB 00:00 2024-04-26T21:43:13.261 INFO:teuthology.orchestra.run.smithi106.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.5 MB/s | 863 kB 00:00 2024-04-26T21:43:13.528 INFO:teuthology.orchestra.run.smithi106.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-04-26T21:43:13.670 INFO:teuthology.orchestra.run.smithi106.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-26T21:43:13.703 INFO:teuthology.orchestra.run.smithi106.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-26T21:43:13.728 INFO:teuthology.orchestra.run.smithi106.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 335 kB/s | 19 kB 00:00 2024-04-26T21:43:13.837 INFO:teuthology.orchestra.run.smithi106.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-26T21:43:13.921 INFO:teuthology.orchestra.run.smithi106.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-26T21:43:14.021 INFO:teuthology.orchestra.run.smithi106.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 680 kB/s | 67 kB 00:00 2024-04-26T21:43:14.130 INFO:teuthology.orchestra.run.smithi106.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T21:43:14.313 INFO:teuthology.orchestra.run.smithi106.stdout:(111/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-04-26T21:43:14.397 INFO:teuthology.orchestra.run.smithi106.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-26T21:43:14.439 INFO:teuthology.orchestra.run.smithi106.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-04-26T21:43:14.464 INFO:teuthology.orchestra.run.smithi106.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 657 kB/s | 43 kB 00:00 2024-04-26T21:43:14.489 INFO:teuthology.orchestra.run.smithi106.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 652 kB/s | 32 kB 00:00 2024-04-26T21:43:14.515 INFO:teuthology.orchestra.run.smithi106.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2024-04-26T21:43:14.598 INFO:teuthology.orchestra.run.smithi106.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T21:43:14.632 INFO:teuthology.orchestra.run.smithi106.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-26T21:43:14.691 INFO:teuthology.orchestra.run.smithi106.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T21:43:14.716 INFO:teuthology.orchestra.run.smithi106.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 427 kB/s | 35 kB 00:00 2024-04-26T21:43:14.766 INFO:teuthology.orchestra.run.smithi106.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T21:43:14.833 INFO:teuthology.orchestra.run.smithi106.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-26T21:43:14.886 INFO:teuthology.orchestra.run.smithi106.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 216 kB/s | 11 kB 00:00 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi106.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 307 kB/s | 18 kB 00:00 2024-04-26T21:43:14.992 INFO:teuthology.orchestra.run.smithi106.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2024-04-26T21:43:15.051 INFO:teuthology.orchestra.run.smithi106.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-04-26T21:43:15.076 INFO:teuthology.orchestra.run.smithi106.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi106.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-26T21:43:15.144 INFO:teuthology.orchestra.run.smithi106.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-26T21:43:15.252 INFO:teuthology.orchestra.run.smithi106.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 732 kB/s | 79 kB 00:00 2024-04-26T21:43:15.302 INFO:teuthology.orchestra.run.smithi106.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-26T21:43:15.377 INFO:teuthology.orchestra.run.smithi106.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-26T21:43:15.411 INFO:teuthology.orchestra.run.smithi106.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 500 kB/s | 16 kB 00:00 2024-04-26T21:43:15.453 INFO:teuthology.orchestra.run.smithi106.stdout:(134/152): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-04-26T21:43:15.487 INFO:teuthology.orchestra.run.smithi106.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T21:43:15.513 INFO:teuthology.orchestra.run.smithi106.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-04-26T21:43:15.538 INFO:teuthology.orchestra.run.smithi106.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2024-04-26T21:43:15.554 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:15.580 INFO:teuthology.orchestra.run.smithi106.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-26T21:43:15.597 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:15.613 INFO:teuthology.orchestra.run.smithi106.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-26T21:43:15.672 INFO:teuthology.orchestra.run.smithi106.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 392 kB/s | 36 kB 00:00 2024-04-26T21:43:15.697 INFO:teuthology.orchestra.run.smithi106.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 617 kB/s | 51 kB 00:00 2024-04-26T21:43:15.739 INFO:teuthology.orchestra.run.smithi106.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-26T21:43:15.781 INFO:teuthology.orchestra.run.smithi106.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-26T21:43:16.065 INFO:teuthology.orchestra.run.smithi106.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.3 MB/s | 427 kB 00:00 2024-04-26T21:43:16.090 INFO:teuthology.orchestra.run.smithi106.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 65 kB/s | 20 kB 00:00 2024-04-26T21:43:16.140 INFO:teuthology.orchestra.run.smithi106.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T21:43:16.242 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:16.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:43:16.465 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:43:16.465 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:43:16.465 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:43:16.465 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:16.490 INFO:teuthology.orchestra.run.smithi106.stdout:(147/152): librados2-19.0.0-3306.gbc612ffb.el9. 9.7 MB/s | 3.4 MB 00:00 2024-04-26T21:43:16.502 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:43:16.707 INFO:teuthology.orchestra.run.smithi106.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-04-26T21:43:16.799 INFO:teuthology.orchestra.run.smithi106.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-26T21:43:19.091 INFO:teuthology.orchestra.run.smithi106.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 580 kB/s | 1.3 MB 00:02 2024-04-26T21:43:20.566 INFO:teuthology.orchestra.run.smithi106.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 651 kB/s | 2.4 MB 00:03 2024-04-26T21:43:29.399 INFO:teuthology.orchestra.run.smithi106.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 792 kB/s | 16 MB 00:21 2024-04-26T21:43:29.408 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:43:29.408 INFO:teuthology.orchestra.run.smithi106.stdout:Total 3.0 MB/s | 222 MB 01:13 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-04-26T21:43:29.732 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:29.978 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:43:30.025 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:43:30.025 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:30.025 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:43:30.026 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:31.645 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:43:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:31.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:32.338 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-26T21:43:32.529 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:43:32.580 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-26T21:43:32.581 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-26T21:43:32.601 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:43:32.602 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:32.602 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:43:32.602 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:43:32.602 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:43:32.602 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:32.729 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:43:32.790 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:43:32.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:43:33.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:43:33.151 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:43:33.236 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:43:33.337 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:43:33.504 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:43:33.562 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:43:33.600 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:43:33.632 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:43:34.303 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:43:34.353 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:43:34.411 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:43:34.519 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:43:34.546 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:43:35.339 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-26T21:43:35.339 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-26T21:43:35.861 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:43:35.931 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:43:37.038 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:43:37.081 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:43:37.229 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:43:37.405 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:43:37.442 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:43:37.525 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:37.609 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:43:37.648 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:43:37.794 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:43:37.862 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:43:38.878 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:43:38.912 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:43:38.984 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:39.038 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:43:39.122 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:43:39.122 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:39.122 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:43:39.122 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:39.205 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:43:39.309 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:43:39.785 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:43:39.785 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:43:39.785 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:43:39.785 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:43:39.785 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:43:39.786 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:41.871 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-26T21:43:41.980 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:43:42.060 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:43:42.183 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:43:42.250 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:43:42.282 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:43:42.407 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:43:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:43:42.982 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:43:43.154 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:43:43.189 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:43:43.315 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:43:43.382 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:43:43.762 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:43:43.827 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:43:43.852 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:43:44.002 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:43:44.053 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:43:44.206 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:43:44.245 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:43:44.320 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:43:44.649 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:43:44.713 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:43:44.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:43:44.902 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:43:44.915 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:43:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:43:45.031 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:43:45.050 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:43:45.079 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:43:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:43:45.133 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:43:45.231 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:43:45.300 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:43:45.365 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:43:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:43:45.453 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:43:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:43:45.545 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:43:45.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:43:45.579 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:43:45.640 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:43:45.679 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:43:45.737 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:43:45.846 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:43:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:43:45.975 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:43:46.015 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:43:46.138 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:43:46.204 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:43:46.312 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:43:46.497 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:43:46.499 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:43:46.552 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:43:46.562 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:43:46.619 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:43:47.247 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:43:47.358 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:43:47.432 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:43:47.485 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:43:47.533 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:43:47.599 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:43:47.884 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:43:47.951 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:43:47.968 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:43:48.035 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:43:48.199 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:43:48.203 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:43:48.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:43:48.298 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:43:48.337 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:43:48.383 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:43:48.398 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:43:48.423 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:43:48.424 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:43:48.449 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:43:48.558 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:43:48.606 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:43:48.607 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:43:48.637 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:43:48.694 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:43:48.731 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:43:48.747 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:43:48.774 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:43:48.876 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:43:48.904 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:43:48.956 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:43:49.004 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:43:49.037 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:43:49.070 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:43:49.179 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:43:49.179 INFO:teuthology.orchestra.run.smithi204.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:43:49.179 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:49.256 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:43:49.435 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:43:49.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:43:49.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:43:49.563 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:43:49.621 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:43:49.690 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:43:49.739 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:43:49.785 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:43:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:43:49.909 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:43:49.967 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:43:50.144 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:43:50.220 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:43:50.267 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:43:50.339 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:43:50.515 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:43:50.587 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:43:51.544 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:43:51.612 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:43:51.644 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:43:51.773 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:43:51.838 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:43:51.970 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:43:52.012 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:43:52.245 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:43:52.276 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:43:52.382 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:43:53.204 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:43:53.339 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:43:54.918 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:43:55.016 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:43:55.057 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:43:55.109 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:43:55.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:43:55.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:43:55.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:43:55.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:43:55.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:43:55.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:43:55.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:43:55.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:43:55.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:43:55.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:43:55.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:43:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:43:55.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:43:55.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:43:55.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:43:55.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:43:55.179 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:43:55.494 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:43:55.555 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:43:55.685 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:43:55.718 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:43:55.776 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:43:56.322 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:43:56.356 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:43:56.457 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:43:56.483 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:43:57.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:43:57.650 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.651 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:43:57.652 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:43:57.653 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:43:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:43:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:43:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:43:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:43:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:43:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:43:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:43:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:43:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:43:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:43:57.663 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-04-26T21:43:58.249 DEBUG:teuthology.parallel:result is None 2024-04-26T21:43:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:59.255 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:59.916 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:43:59.986 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:44:00.149 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:44:00.149 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:44:00.149 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:44:00.149 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:00.192 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:44:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-26T21:44:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:13.938 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:44:13.985 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:44:13.985 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:13.985 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:44:13.985 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:15.634 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:44:15.714 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:44:15.715 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:15.715 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:44:15.715 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:44:15.715 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:44:15.715 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:16.051 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:44:16.122 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:44:16.122 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:16.122 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:44:16.122 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:44:16.123 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:44:16.123 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:16.610 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:44:16.679 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:16.810 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:44:16.873 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:44:16.946 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:44:17.177 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:44:17.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:44:17.316 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:44:17.424 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:44:17.589 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:44:17.648 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:44:17.688 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:44:17.721 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:44:18.384 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:44:18.433 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:44:18.490 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:44:18.627 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:44:18.659 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:44:19.927 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:44:19.970 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:44:21.065 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:44:21.125 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:44:21.270 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:44:21.450 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:44:21.505 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:44:21.575 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:44:21.575 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:44:21.575 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:21.631 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:44:21.670 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:44:21.782 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:44:21.850 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:44:22.874 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:44:22.908 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:44:22.978 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:23.026 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:44:23.107 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:44:23.107 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:23.107 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:44:23.107 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:23.201 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:44:23.301 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:44:23.302 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:23.302 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:44:23.302 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:44:23.302 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:44:23.302 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:44:23.804 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:44:23.805 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:44:23.805 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:44:23.805 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:44:23.805 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:44:23.805 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:28.850 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:44:28.919 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:44:29.002 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:44:29.044 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:44:29.093 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:44:29.423 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:44:29.480 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:44:29.507 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:44:30.455 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:44:30.505 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:44:30.567 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:44:31.912 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:44:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:44:32.091 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:44:32.263 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:44:32.310 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:44:32.310 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:44:32.428 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:44:32.467 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:44:32.467 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:44:32.641 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:44:32.771 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:44:33.062 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:44:33.063 INFO:teuthology.orchestra.run.smithi106.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:44:33.063 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:44:38.762 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:44:38.763 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:44:38.767 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:44:38.767 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:44:38.767 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:44:38.767 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:44:41.274 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:44:41.274 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.275 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.276 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:44:41.277 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:44:41.278 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:44:41.279 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:44:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:44:41.281 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:44:41.282 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:44:41.283 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.284 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:44:41.285 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:44:41.286 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:44:41.287 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:44:41.287 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:41.287 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-26T21:44:41.892 DEBUG:teuthology.parallel:result is None 2024-04-26T21:44:41.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:44:42.192 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:44:42.243 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:44:42.244 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:44:42.244 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:44:42.245 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:44:42.385 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:44:42.457 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:44:42.457 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:44:42.457 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:44:42.459 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:44:42.459 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:42.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:44:42.539 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:42.539 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:44:42.626 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:44:42.626 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:42.626 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:44:42.688 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:44:42.795 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:42.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:44:42.886 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:44:42.956 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:44:42.957 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:42.957 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:44:43.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:44:43.125 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:43.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:44:43.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:44:43.323 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:44:43.323 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:43.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:44:43.393 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:44:43.504 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:43.504 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:44:43.580 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:44:43.693 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T21:44:43.776 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T21:44:43.776 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2024-04-26T21:44:43.781 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-04-26T21:44:43.857 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T21:44:43.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:44:43.861 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:44:43.923 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T21:44:43.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:44:43.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:44:43.997 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T21:44:43.997 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T21:44:43.997 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T21:44:43.998 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:43.998 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:44:44.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:44.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:44.287 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:44.287 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:44:44.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:44.564 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:44.644 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T21:44:44.644 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T21:44:44.644 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7675230', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi106.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGvZg1Zby3QB9t81i0ElWC414PbVAsNiey83rUHUZCko6B8eS6pLsKEIMacb1QCNwzf/k21snr7aND/AUTJfSgk=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvsDwOjxTbut750N71/3tg1jCt0r43pR8LS3tiL6W+phHCPmd1t7l2VKVcjeP7Oe+yPoy4+oW+C+So7jUXfp1M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T21:44:44.645 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:44:44.649 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:44:44.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:44:44.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:44:44.812 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:44.812 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:44.849 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:44.849 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:44.928 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T21:44:44.928 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:44.929 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:44.929 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-26 21:44:33.490278148 +0000 2024-04-26T21:44:44.929 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-26 21:41:18.531673478 +0000 2024-04-26T21:44:44.929 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-26 21:41:18.531673478 +0000 2024-04-26T21:44:44.929 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-26 21:41:18.531673478 +0000 2024-04-26T21:44:44.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:45.026 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-26T21:44:45.026 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-26T21:44:45.026 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000338829 s, 1.5 MB/s 2024-04-26T21:44:45.029 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:45.069 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:45.148 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-26 21:44:33.490278148 +0000 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-26 21:41:19.126659444 +0000 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-26 21:41:19.126659444 +0000 2024-04-26T21:44:45.149 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-26 21:41:19.126659444 +0000 2024-04-26T21:44:45.149 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:45.245 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-26T21:44:45.245 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-26T21:44:45.245 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000264465 s, 1.9 MB/s 2024-04-26T21:44:45.248 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:45.289 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:45.367 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:45.367 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:45.367 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T21:44:45.367 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:45.368 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:45.368 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-26 21:44:33.491278124 +0000 2024-04-26T21:44:45.368 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-26 21:41:19.748644772 +0000 2024-04-26T21:44:45.368 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-26 21:41:19.748644772 +0000 2024-04-26T21:44:45.368 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-26 21:41:19.748644772 +0000 2024-04-26T21:44:45.368 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:45.466 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-26T21:44:45.467 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-26T21:44:45.467 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000285408 s, 1.8 MB/s 2024-04-26T21:44:45.471 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:45.512 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:45.589 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:45.589 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:45.589 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T21:44:45.589 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:45.590 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:45.590 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-26 21:44:33.492278101 +0000 2024-04-26T21:44:45.590 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-26 21:41:20.333630974 +0000 2024-04-26T21:44:45.590 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-26 21:41:20.333630974 +0000 2024-04-26T21:44:45.590 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-26 21:41:20.333630974 +0000 2024-04-26T21:44:45.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:45.689 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-26T21:44:45.689 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-26T21:44:45.689 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000276213 s, 1.9 MB/s 2024-04-26T21:44:45.692 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:45.732 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:45.732 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:45.733 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:45.789 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:45.789 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:45.837 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:43:49.634336791 +0000 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 21:41:18.499598139 +0000 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 21:41:18.499598139 +0000 2024-04-26T21:44:45.838 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 21:41:18.499598139 +0000 2024-04-26T21:44:45.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:45.909 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:44:45.909 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:44:45.909 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000350872 s, 1.5 MB/s 2024-04-26T21:44:45.912 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:45.966 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:43:49.635336779 +0000 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 21:41:19.091590724 +0000 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 21:41:19.091590724 +0000 2024-04-26T21:44:46.016 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 21:41:19.091590724 +0000 2024-04-26T21:44:46.016 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:46.090 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:44:46.090 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:44:46.090 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000303446 s, 1.7 MB/s 2024-04-26T21:44:46.094 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:46.149 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:46.198 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:43:49.635336779 +0000 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 21:41:19.666583522 +0000 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 21:41:19.666583522 +0000 2024-04-26T21:44:46.199 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 21:41:19.666583522 +0000 2024-04-26T21:44:46.199 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:46.276 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:44:46.276 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:44:46.276 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00035829 s, 1.4 MB/s 2024-04-26T21:44:46.280 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:46.334 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:46.384 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:46.384 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:46.384 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T21:44:46.384 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:46.384 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:46.385 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:43:49.636336766 +0000 2024-04-26T21:44:46.385 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 21:41:20.284575781 +0000 2024-04-26T21:44:46.385 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 21:41:20.284575781 +0000 2024-04-26T21:44:46.385 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 21:41:20.284575781 +0000 2024-04-26T21:44:46.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:46.460 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:44:46.460 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:44:46.460 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000305808 s, 1.7 MB/s 2024-04-26T21:44:46.464 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:46.517 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:46.517 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:44:46.517 INFO:tasks.ceph:Generating config... 2024-04-26T21:44:46.518 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T21:44:46.518 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T21:44:46.518 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T21:44:46.518 INFO:tasks.ceph:[client] pid file = 2024-04-26T21:44:46.518 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T21:44:46.519 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T21:44:46.519 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T21:44:46.520 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T21:44:46.520 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T21:44:46.520 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T21:44:46.520 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T21:44:46.520 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:44:46.521 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T21:44:46.521 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T21:44:46.521 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T21:44:46.521 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T21:44:46.521 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T21:44:46.521 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T21:44:46.521 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T21:44:46.621 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T21:44:46.628 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T21:44:46.717 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:44:46.784 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.106'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-04-26T21:44:46.785 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T21:44:46.786 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:46.786 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:44:46.833 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid e6ee84ed-cbe5-4ff1-aa0d-97a291446523 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = quincy 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:fsid e6ee84ed-cbe5-4ff1-aa0d-97a291446523 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2024-04-26T21:44:46.961544+0000 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-26T21:44:46.961544+0000 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 17 (quincy) 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-26T21:44:46.961 INFO:teuthology.orchestra.run.smithi106.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-04-26T21:44:46.962 INFO:teuthology.orchestra.run.smithi106.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-04-26T21:44:46.962 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T21:44:46.964 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:44:47.001 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e6ee84ed-cbe5-4ff1-aa0d-97a291446523... 2024-04-26T21:44:47.002 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:44:47.047 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:44:47.169 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-26T21:44:47.169 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-26T21:44:47.169 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:44:47.169 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-26T21:44:47.169 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-26T21:44:47.170 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:44:47.171 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = e6ee84ed-cbe5-4ff1-aa0d-97a291446523 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.172 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.173 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-26T21:44:47.174 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: bdev async discard = True 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: bdev enable discard = True 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore allocator = bitmap 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 1/20 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 1/20 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 4/10 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:44:47.175 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: debug finisher = 20 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.176 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:44:47.177 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-26T21:44:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-26T21:44:47.179 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-26T21:44:47.180 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-26T21:44:47.190 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-04-26T21:44:47.190 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-04-26T21:44:47.190 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-04-26T21:44:47.191 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:44:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.106,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = e6ee84ed-cbe5-4ff1-aa0d-97a291446523 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.193 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-04-26T21:44:47.194 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-04-26T21:44:47.195 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore allocator = bitmap 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 1/20 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 1/20 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 4/10 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:44:47.196 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-04-26T21:44:47.197 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: debug finisher = 20 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:44:47.198 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-04-26T21:44:47.199 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-04-26T21:44:47.200 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-04-26T21:44:47.201 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-04-26T21:44:47.224 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T21:44:47.224 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T21:44:47.343 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T21:44:47.343 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:47.343 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T21:44:47.381 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:47.381 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T21:44:47.462 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:44:47.462 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:47.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:44:47.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:44:47.592 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:44:47.592 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:47.675 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:44:47.676 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:47.676 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:44:47.676 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:44:47.809 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:44:47.809 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:47.908 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T21:44:47.908 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T21:44:47.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:44:48.039 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:44:48.046 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:44:48.216 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:44:48.226 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T21:44:48.226 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:44:48.344 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:44:48.350 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T21:44:48.418 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:44:48.548 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:44:48.615 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T21:44:48.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:44:48.857 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:44:48.865 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T21:44:48.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:44:49.109 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:44:49.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T21:44:49.201 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T21:44:49.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:44:49.324 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T21:44:49.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T21:44:49.454 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T21:44:49.501 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T21:44:49.623 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T21:44:49.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T21:44:49.775 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T21:44:49.812 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T21:44:49.812 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:44:49.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T21:44:49.892 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:49.892 INFO:tasks.ceph:role: osd.0 2024-04-26T21:44:49.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:44:49.892 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:44:49.975 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:44:50.385 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-26T21:44:50.392 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-26T21:44:50.392 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T21:44:50.541 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T21:44:50.629 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T21:44:50.698 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:50.698 INFO:tasks.ceph:role: osd.1 2024-04-26T21:44:50.699 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:44:50.699 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:44:51.136 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-26T21:44:51.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-26T21:44:51.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T21:44:51.248 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T21:44:51.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T21:44:51.396 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:51.397 INFO:tasks.ceph:role: osd.2 2024-04-26T21:44:51.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:44:51.397 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:44:51.485 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:44:51.485 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:44:51.486 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:44:51.879 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-26T21:44:51.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-26T21:44:51.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T21:44:52.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T21:44:52.084 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T21:44:52.161 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:44:52.161 INFO:tasks.ceph:role: osd.3 2024-04-26T21:44:52.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:44:52.161 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:44:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:44:52.652 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-26T21:44:52.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-26T21:44:52.658 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T21:44:52.822 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T21:44:52.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.031+0000 7fd88872c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T21:44:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.032+0000 7fd88872c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T21:44:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.032+0000 7fd88872c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:44:53.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.032+0000 7fd88872c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:44:53.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.032+0000 7fd88872c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:44:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:53.036+0000 7fd88872c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T21:44:55.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T21:44:56.071 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:56.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.193+0000 7fdf1445d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T21:44:56.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.194+0000 7fdf1445d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T21:44:56.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.194+0000 7fdf1445d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:44:56.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.194+0000 7fdf1445d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:44:56.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.194+0000 7fdf1445d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:44:56.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:56.197+0000 7fdf1445d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T21:44:59.145 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T21:44:59.223 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:44:59.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.348+0000 7fdea43b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T21:44:59.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.348+0000 7fdea43b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T21:44:59.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.349+0000 7fdea43b6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:44:59.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.349+0000 7fdea43b6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:44:59.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.349+0000 7fdea43b6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:44:59.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:44:59.351+0000 7fdea43b6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T21:45:02.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T21:45:02.407 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:02.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.536+0000 7fa762c23640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T21:45:02.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.536+0000 7fa762c23640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T21:45:02.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.536+0000 7fa762c23640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:45:02.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.536+0000 7fa762c23640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:45:02.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.536+0000 7fa762c23640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:45:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:02.539+0000 7fa762c23640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T21:45:05.494 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T21:45:05.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T21:45:05.663 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:45:05.663 INFO:tasks.ceph:role: osd.4 2024-04-26T21:45:05.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:45:05.663 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:45:06.172 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-26T21:45:06.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-26T21:45:06.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T21:45:06.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T21:45:06.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T21:45:06.517 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:45:06.517 INFO:tasks.ceph:role: osd.5 2024-04-26T21:45:06.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:45:06.518 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:45:06.599 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:45:06.600 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:45:06.600 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:45:06.600 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:45:06.600 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:45:06.953 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-26T21:45:06.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-26T21:45:06.959 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T21:45:07.091 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T21:45:07.175 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T21:45:07.249 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:45:07.249 INFO:tasks.ceph:role: osd.6 2024-04-26T21:45:07.249 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:45:07.249 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:45:07.331 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:45:07.331 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:45:07.331 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:45:07.332 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:45:07.723 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-26T21:45:07.728 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-26T21:45:07.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T21:45:07.875 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T21:45:07.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T21:45:08.053 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:45:08.053 INFO:tasks.ceph:role: osd.7 2024-04-26T21:45:08.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:45:08.053 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:45:08.142 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:45:08.143 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:45:08.548 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-26T21:45:08.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-26T21:45:08.556 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T21:45:08.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T21:45:08.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:08.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.906+0000 7fd7c320d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T21:45:08.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.907+0000 7fd7c320d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T21:45:08.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.907+0000 7fd7c320d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:45:08.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.907+0000 7fd7c320d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:45:08.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.907+0000 7fd7c320d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:45:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:08.911+0000 7fd7c320d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T21:45:11.893 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T21:45:11.978 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:12.104 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.104+0000 7f3eb9232640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T21:45:12.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.104+0000 7f3eb9232640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T21:45:12.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.104+0000 7f3eb9232640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:45:12.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.104+0000 7f3eb9232640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:45:12.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.104+0000 7f3eb9232640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:45:12.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:12.107+0000 7f3eb9232640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T21:45:14.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:14.655 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:15.058 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T21:45:15.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:15.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.272+0000 7ff16d89a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T21:45:15.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.273+0000 7ff16d89a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T21:45:15.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.274+0000 7ff16d89a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:45:15.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.274+0000 7ff16d89a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:45:15.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.274+0000 7ff16d89a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:45:15.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:15.276+0000 7ff16d89a640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T21:45:18.477 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T21:45:18.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:18.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.698+0000 7f3ec2caf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T21:45:18.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.699+0000 7f3ec2caf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T21:45:18.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.699+0000 7f3ec2caf640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:45:18.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.699+0000 7f3ec2caf640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:45:18.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.699+0000 7f3ec2caf640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:45:18.702 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:18.701+0000 7f3ec2caf640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T21:45:21.682 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T21:45:21.767 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T21:45:21.768 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:21.768 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T21:45:21.857 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:21.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T21:45:21.933 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:21.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T21:45:22.063 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:22.064 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T21:45:22.151 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:22.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T21:45:22.233 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:22.233 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T21:45:22.308 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:22.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T21:45:22.434 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.434 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T21:45:22.522 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T21:45:22.604 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.604 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T21:45:22.686 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.686 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T21:45:22.770 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.771 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T21:45:22.854 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.854 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T21:45:22.935 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:22.935 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T21:45:23.019 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:23.019 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T21:45:23.076 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:23.076 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T21:45:23.127 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:23.127 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T21:45:23.184 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:23.184 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T21:45:23.238 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T21:45:23.238 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:45:23.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBQICxmyV5pAhAApZSAbzZOWb572E7vzC6aJQ== 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBQICxmnY2XFBAAtaQO3LQwin4e9iNrklJH6w== 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBQICxmLYa9IBAAndFvm1tzi44OqYb6ttR3Sw== 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBVICxmvpP+ARAAApDQrRCTWu21gf/CGyROOg== 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2024-04-26T21:45:23.323 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBYICxmMM6lCxAANvEZrGQcAR75+Xy8ktUM5A== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBbICxmZPvaFBAA/2/DymuSvYvc1WYgY1jO/Q== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBeICxmhYIOIBAAa0ISn+YDALtH4Ma087UI6Q== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBQICxmzUzsDBAAIx87eHODdurs4RGojAb/nQ== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBQICxm684cMxAAYTf8acqdf2CCeHokqOS0Xg== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxm90qUBhAA9teVVNbkqfjkCWMBcZK//Q== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBkICxmTlgVNhAA3FMFqR8eN9aQooYqEN2kxQ== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBoICxmoSRBBhAAVLMhNepkCpcdbgJWdcr8FA== 2024-04-26T21:45:23.324 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBrICxm7HdTEBAA5myIxNTNd3FaNjEbMiILEQ== 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBuICxmrWiyKRAAxRDKvFrBA1YRdRif8ZH/ow== 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q== 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout:[client.2] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmCucaGxAAbxoXORUIaoRjav5XMYwKZA== 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmjz8uJRAA7zXqZtFgUCBE9tb+7barEw== 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout:[client.3] 2024-04-26T21:45:23.325 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmsp5GLhAAEuL2LpnxmDBCfgz5b3AcuQ== 2024-04-26T21:45:23.327 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-04-26T21:45:23.327 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBQICxmyV5pAhAApZSAbzZOWb572E7vzC6aJQ== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBQICxmnY2XFBAAtaQO3LQwin4e9iNrklJH6w== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBQICxmLYa9IBAAndFvm1tzi44OqYb6ttR3Sw== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBVICxmvpP+ARAAApDQrRCTWu21gf/CGyROOg== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBYICxmMM6lCxAANvEZrGQcAR75+Xy8ktUM5A== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBbICxmZPvaFBAA/2/DymuSvYvc1WYgY1jO/Q== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBeICxmhYIOIBAAa0ISn+YDALtH4Ma087UI6Q== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBQICxmzUzsDBAAIx87eHODdurs4RGojAb/nQ== 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-04-26T21:45:23.328 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBQICxm684cMxAAYTf8acqdf2CCeHokqOS0Xg== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBRICxm90qUBhAA9teVVNbkqfjkCWMBcZK//Q== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBkICxmTlgVNhAA3FMFqR8eN9aQooYqEN2kxQ== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBoICxmoSRBBhAAVLMhNepkCpcdbgJWdcr8FA== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBrICxm7HdTEBAA5myIxNTNd3FaNjEbMiILEQ== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBuICxmrWiyKRAAxRDKvFrBA1YRdRif8ZH/ow== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[client.2] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBRICxmCucaGxAAbxoXORUIaoRjav5XMYwKZA== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBRICxmjz8uJRAA7zXqZtFgUCBE9tb+7barEw== 2024-04-26T21:45:23.329 INFO:teuthology.orchestra.run.smithi204.stdout:[client.3] 2024-04-26T21:45:23.330 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQBRICxmsp5GLhAAEuL2LpnxmDBCfgz5b3AcuQ== 2024-04-26T21:45:23.333 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:45:23.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:45:23.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:23.519 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:23.657 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:23.663 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:23.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:23.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:23.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:23.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.069 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.204 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.211 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:45:24.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:45:24.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:24.485 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:24.620 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:24.627 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:45:24.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.762 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.889 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:24.897 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:25.032 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:25.038 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:25.167 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:25.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:45:25.306 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.312 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.447 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.602 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.740 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:45:25.877 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T21:45:25.877 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T21:45:25.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:45:26.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T21:45:26.169 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T21:45:26.251 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:45:26.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T21:45:26.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T21:45:26.546 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:45:26.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T21:45:26.754 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:26.761 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:45:26.811 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T21:45:26.811 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T21:45:26.811 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T21:45:26.816 INFO:tasks.ceph.mon.a:Started 2024-04-26T21:45:26.816 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T21:45:26.817 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T21:45:26.821 INFO:tasks.ceph.mon.b:Started 2024-04-26T21:45:26.821 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T21:45:26.822 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T21:45:26.826 INFO:tasks.ceph.mon.c:Started 2024-04-26T21:45:26.826 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T21:45:26.826 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T21:45:26.826 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T21:45:26.831 INFO:tasks.ceph.mgr.y:Started 2024-04-26T21:45:26.831 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T21:45:26.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T21:45:26.835 INFO:tasks.ceph.mgr.x:Started 2024-04-26T21:45:26.835 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T21:45:26.835 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.005+0000 7fd97be00640 1 Processor -- start 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.005+0000 7fd97be00640 1 -- start start 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.006+0000 7fd97be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.006+0000 7fd97be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.006+0000 7fd97be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c0d0170 con 0x7fd97c15fbd0 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.006+0000 7fd97be00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c0d0450 con 0x7fd97c0d2c50 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.006+0000 7fd97be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c0d0730 con 0x7fd97c15f7f0 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56168/0 (socket says 172.21.15.106:56168) 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 learned_addr learned my addr 172.21.15.106:0/3713130501 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.007+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.208+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.208+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.208+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.208+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.208+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.209+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.209+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.209+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.209+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.209+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:45:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.610+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.611+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:45:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:27.611+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.411+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.411+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:28.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:28.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:45:28.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:28.412+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:45:30.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 msgr2=0x7fd97c0cfb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.204:3300/0 conn(0x7fd97c15f7f0 0x7fd97c0cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 -- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 msgr2=0x7fd97c0d30c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 --2- 172.21.15.106:0/3713130501 >> v2:172.21.15.106:3300/0 conn(0x7fd97c0d2c50 0x7fd97c0d30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 -- 172.21.15.106:0/3713130501 >> v1:172.21.15.204:6789/0 conn(0x7fd97c15fbd0 legacy=0x7fd97c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 -- 172.21.15.106:0/3713130501 --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c0d0a10 con 0x7fd95c003690 2024-04-26T21:45:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.005+0000 7fd973e00640 1 -- 172.21.15.106:0/3713130501 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0d0450 con 0x7fd95c0072e0 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.006+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.007+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.007+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.007+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.007+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:30.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.208+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.209+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:30.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.209+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:30.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.209+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:30.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.209+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:45:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.610+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.611+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.611+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.611+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:30.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:30.611+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:45:31.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:31.412+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:31.413+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:31.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:31.413+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:31.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:31.413+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:31.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:31.413+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:45:32.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.044+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3975285956 0 0) 0x7fd97c0d0a10 con 0x7fd95c003690 2024-04-26T21:45:32.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.044+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd960003520 con 0x7fd95c003690 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2780111266 0 0) 0x7fd960003520 con 0x7fd95c003690 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 msgr2=0x7fd95c009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd95c0072e0 0x7fd95c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c00afb0 con 0x7fd95c003690 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fd96c00d580 con 0x7fd95c003690 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 >> v1:172.21.15.106:6789/0 conn(0x7fd95c003690 legacy=0x7fd95c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd97b400640 1 -- 172.21.15.106:0/3713130501 reap_dead start 2024-04-26T21:45:32.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9600053c0 0x7fd96000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0d0450 con 0x7fd960004800 2024-04-26T21:45:32.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd96000c2a0 con 0x7fd9600053c0 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.046+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd96000c4a0 con 0x7fd960003a20 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.047+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.047+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 msgr2=0x7fd960003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97aa00640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 msgr2=0x7fd960003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:32.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97b400640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9600053c0 0x7fd96000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97aa00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 msgr2=0x7fd960003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9600053c0 msgr2=0x7fd96000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9600053c0 0x7fd96000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.048+0000 7fd97a000640 1 -- 172.21.15.106:0/3713130501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd960003520 con 0x7fd960004800 2024-04-26T21:45:32.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.050+0000 7fd97a000640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd960004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd968007c30 tx=0x7fd960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b1a511ecd5bb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.051+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968034070 con 0x7fd960004800 2024-04-26T21:45:32.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.051+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd968007020 con 0x7fd960004800 2024-04-26T21:45:32.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.051+0000 7fd979600640 1 -- 172.21.15.106:0/3713130501 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680051f0 con 0x7fd960004800 2024-04-26T21:45:32.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 -- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 msgr2=0x7fd960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd960004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd968007c30 tx=0x7fd960005830 comp rx=0 tx=0).stop 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 -- 172.21.15.106:0/3713130501 shutdown_connections 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9600053c0 0x7fd96000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd960004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 --2- 172.21.15.106:0/3713130501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd960003a20 0x7fd960003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 -- 172.21.15.106:0/3713130501 >> 172.21.15.106:0/3713130501 conn(0x7fd97c0d4f80 msgr2=0x7fd97c0d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 -- 172.21.15.106:0/3713130501 shutdown_connections 2024-04-26T21:45:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.052+0000 7fd97be00640 1 -- 172.21.15.106:0/3713130501 wait complete. 2024-04-26T21:45:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.057+0000 7fd97be00640 1 Processor -- start 2024-04-26T21:45:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.057+0000 7fd97be00640 1 -- start start 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd960003a20 0x7fd97c0b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c15f430 con 0x7fd960004800 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c160050 con 0x7fd960003a20 2024-04-26T21:45:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c15f5b0 con 0x7fd9600053c0 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53806/0 (socket says 172.21.15.106:53806) 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97a000640 1 -- 172.21.15.106:0/2527857297 learned_addr learned my addr 172.21.15.106:0/2527857297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97b400640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.059+0000 7fd97aa00640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd960003a20 0x7fd97c0b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97b400640 1 -- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 msgr2=0x7fd97c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97b400640 1 -- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 msgr2=0x7fd97c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97b400640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97b400640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 -- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 msgr2=0x7fd97c158aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 -- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd960003a20 msgr2=0x7fd97c0b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd960003a20 0x7fd97c0b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 -- 172.21.15.106:0/2527857297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0cc890 con 0x7fd960004800 2024-04-26T21:45:32.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.060+0000 7fd97a000640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9680055b0 tx=0x7fd968001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.061+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968034050 con 0x7fd960004800 2024-04-26T21:45:32.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.061+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9680051d0 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.061+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1f8b40 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.061+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680110b0 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.061+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1f9100 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.062+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd968042670 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.062+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd968030780 con 0x7fd960004800 2024-04-26T21:45:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.063+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c0693c0 con 0x7fd960004800 2024-04-26T21:45:32.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.066+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd968042aa0 con 0x7fd960004800 2024-04-26T21:45:32.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.141+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd97c08ea90 con 0x7fd960004800 2024-04-26T21:45:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.143+0000 7fd973400640 1 -- 172.21.15.106:0/2527857297 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd968042c80 con 0x7fd960004800 2024-04-26T21:45:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 msgr2=0x7fd97c1f82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9680055b0 tx=0x7fd968001190 comp rx=0 tx=0).stop 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 shutdown_connections 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9600053c0 0x7fd97c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd960004800 0x7fd97c1f82f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 --2- 172.21.15.106:0/2527857297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd960003a20 0x7fd97c0b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 >> 172.21.15.106:0/2527857297 conn(0x7fd97c0d4f80 msgr2=0x7fd97c0d5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.145+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 shutdown_connections 2024-04-26T21:45:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:32.146+0000 7fd97be00640 1 -- 172.21.15.106:0/2527857297 wait complete. 2024-04-26T21:45:32.163 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T21:45:32.164 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T21:45:32.164 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:32.164 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T21:45:32.200 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:32.199+0000 7fe6ade18040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:45:32.227 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:32.226+0000 7faf2e91c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:45:32.230 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:32.230 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T21:45:32.349 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:32.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T21:45:32.397 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:32.396+0000 7fe6ade18040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:32.418 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:32.418+0000 7faf2e91c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:32.419 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:45:32.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T21:45:32.466 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:32.462+0000 7fe6ade18040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:45:32.486 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:32.486+0000 7faf2e91c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:45:32.525 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:32.525 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T21:45:32.603 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:32.604 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T21:45:32.719 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:32.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T21:45:32.794 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:32.794+0000 7faf2e91c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:45:32.817 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:32.816+0000 7fe6ade18040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:45:32.830 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:45:32.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T21:45:32.852 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:32.852+0000 7faf2e91c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:45:32.870 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:32.869+0000 7fe6ade18040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:45:32.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d 0 2024-04-26T21:45:33.146 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.145+0000 7fc326600640 1 Processor -- start 2024-04-26T21:45:33.146 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.145+0000 7fc326600640 1 -- start start 2024-04-26T21:45:33.148 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc326600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.148 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc326600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3200709f0 0x7fc32015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.148 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc326600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32015e2e0 con 0x7fc32006c730 2024-04-26T21:45:33.148 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc326600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc32006efb0 con 0x7fc3200709f0 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc326600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc32006f570 con 0x7fc32006c350 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc317e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3200709f0 0x7fc32015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55348/0 (socket says 172.21.15.204:55348) 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc317e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3200709f0 0x7fc32015dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33082/0 (socket says 172.21.15.204:33082) 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 -- 172.21.15.204:0/2961136991 learned_addr learned my addr 172.21.15.204:0/2961136991 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325c00640 1 --1- 172.21.15.204:0/2961136991 >> v1:172.21.15.204:6789/0 conn(0x7fc32006c730 0x7fc32006fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43642/0 (socket says 172.21.15.204:43642) 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 -- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc3200cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 -- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc3200cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.147+0000 7fc325200640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.148+0000 7fc317e00640 1 -- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc3200cffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.148+0000 7fc31fe00640 1 -- 172.21.15.204:0/2961136991 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422008943 0 0) 0x7fc318001490 con 0x7fc32006c730 2024-04-26T21:45:33.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.148+0000 7fc317e00640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.150 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.148+0000 7fc317e00640 1 -- 172.21.15.204:0/2961136991 >> v1:172.21.15.204:6789/0 conn(0x7fc32006c730 legacy=0x7fc32006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.150 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.148+0000 7fc317e00640 1 -- 172.21.15.204:0/2961136991 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200ce660 con 0x7fc3200709f0 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.149+0000 7fc317e00640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc32015dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc30802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcdce2f5a014cff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.149+0000 7fc31fe00640 1 -- 172.21.15.204:0/2961136991 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308037070 con 0x7fc3200709f0 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.149+0000 7fc31fe00640 1 -- 172.21.15.204:0/2961136991 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc308032440 con 0x7fc3200709f0 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.149+0000 7fc31fe00640 1 -- 172.21.15.204:0/2961136991 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308032810 con 0x7fc3200709f0 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 -- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 msgr2=0x7fc32015dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc32015dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc30802a180 comp rx=0 tx=0).stop 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 -- 172.21.15.204:0/2961136991 shutdown_connections 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc32015dda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 --2- 172.21.15.204:0/2961136991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc3200cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 -- 172.21.15.204:0/2961136991 >> 172.21.15.204:0/2961136991 conn(0x7fc3200b38a0 msgr2=0x7fc3200b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.151 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 -- 172.21.15.204:0/2961136991 shutdown_connections 2024-04-26T21:45:33.152 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.150+0000 7fc326600640 1 -- 172.21.15.204:0/2961136991 wait complete. 2024-04-26T21:45:33.152 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.152+0000 7fc326600640 1 Processor -- start 2024-04-26T21:45:33.152 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.152+0000 7fc326600640 1 -- start start 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32006c730 0x7fc320158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc320141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc32006f200 con 0x7fc32006c730 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc3200d0510 con 0x7fc3200709f0 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc326600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc32006cbe0 con 0x7fc32006c350 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55354/0 (socket says 172.21.15.204:55354) 2024-04-26T21:45:33.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 -- 172.21.15.204:0/3360130585 learned_addr learned my addr 172.21.15.204:0/3360130585 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325c00640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc320141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 -- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc320153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 -- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc320153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325200640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc317e00640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32006c730 0x7fc320158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.153+0000 7fc325c00640 1 -- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 msgr2=0x7fc320153680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc325c00640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc325c00640 1 -- 172.21.15.204:0/3360130585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32006c730 msgr2=0x7fc320158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc325c00640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32006c730 0x7fc320158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc325c00640 1 -- 172.21.15.204:0/3360130585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320140de0 con 0x7fc3200709f0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc325c00640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc320141e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc31800c6e0 tx=0x7fc31800cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318018070 con 0x7fc3200709f0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc318004d60 con 0x7fc3200709f0 2024-04-26T21:45:33.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3201410c0 con 0x7fc3200709f0 2024-04-26T21:45:33.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31800a880 con 0x7fc3200709f0 2024-04-26T21:45:33.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc320141680 con 0x7fc3200709f0 2024-04-26T21:45:33.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc31801b070 con 0x7fc3200709f0 2024-04-26T21:45:33.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.154+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc31801d780 con 0x7fc3200709f0 2024-04-26T21:45:33.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.155+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc320069ad0 con 0x7fc3200709f0 2024-04-26T21:45:33.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.160+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc32015e2e0 con 0x7fc3200709f0 2024-04-26T21:45:33.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.242+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d", "id": 0} v 0) -- 0x7fc3200d09f0 con 0x7fc3200709f0 2024-04-26T21:45:33.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.249+0000 7fc31d600640 1 -- 172.21.15.204:0/3360130585 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc31800aac0 con 0x7fc3200709f0 2024-04-26T21:45:33.250 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-04-26T21:45:33.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 msgr2=0x7fc320141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc320141e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc31800c6e0 tx=0x7fc31800cba0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 shutdown_connections 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3200709f0 0x7fc320141e80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32006c730 0x7fc320158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 --2- 172.21.15.204:0/3360130585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32006c350 0x7fc320153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 >> 172.21.15.204:0/3360130585 conn(0x7fc3200b38a0 msgr2=0x7fc32015dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 shutdown_connections 2024-04-26T21:45:33.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.251+0000 7fc326600640 1 -- 172.21.15.204:0/3360130585 wait complete. 2024-04-26T21:45:33.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 2434bd98-00f9-476b-9fc9-1d32817f20a1 1 2024-04-26T21:45:33.446 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.445+0000 7f31ace00640 1 Processor -- start 2024-04-26T21:45:33.446 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.445+0000 7f31ace00640 1 -- start start 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.446+0000 7f31ace00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31a80d3780 0x7f31a80d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.446+0000 7f31ace00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a80d25f0 con 0x7f31a80b3510 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.446+0000 7f31ace00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a80d40a0 con 0x7f31a80d58b0 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.446+0000 7f31ace00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f31a80b2070 con 0x7f31a80d3780 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.447+0000 7f31a5400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f31a80d58b0 0x7f31a80d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43654/0 (socket says 172.21.15.204:43654) 2024-04-26T21:45:33.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.447+0000 7f31a5400640 1 -- 172.21.15.204:0/2672757786 learned_addr learned my addr 172.21.15.204:0/2672757786 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:33.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.447+0000 7f31a5e00640 1 --2- 172.21.15.204:0/2672757786 >> v2:172.21.15.106:3300/0 conn(0x7f31a80d3780 0x7f31a80d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.447+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595521515 0 0) 0x7f31a80d40a0 con 0x7f31a80d58b0 2024-04-26T21:45:33.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.447+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f318c003520 con 0x7f31a80d58b0 2024-04-26T21:45:33.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2364528468 0 0) 0x7f318c003520 con 0x7f31a80d58b0 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 >> v2:172.21.15.106:3300/0 conn(0x7f31a80d3780 msgr2=0x7f31a80d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 --2- 172.21.15.204:0/2672757786 >> v2:172.21.15.106:3300/0 conn(0x7f31a80d3780 0x7f31a80d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 >> v1:172.21.15.106:6789/0 conn(0x7f31a80b3510 legacy=0x7f31a80d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a80d0960 con 0x7f31a80d58b0 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f3194001670 con 0x7f31a80d58b0 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 >> v1:172.21.15.204:6789/0 conn(0x7f31a80d58b0 legacy=0x7f31a80d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 0x7f318c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f318c004800 0x7f318c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0051c0 0x7f318c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80b2070 con 0x7f318c0051c0 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f318c00c060 con 0x7f318c0039b0 2024-04-26T21:45:33.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f318c00c260 con 0x7f318c004800 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a5400640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f318c004800 0x7f318c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a5e00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 0x7f318c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.448+0000 7f31a6800640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0051c0 0x7f318c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a5e00640 1 -- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f318c004800 msgr2=0x7f318c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a5e00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f318c004800 0x7f318c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a5e00640 1 -- 172.21.15.204:0/2672757786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0051c0 msgr2=0x7f318c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a5e00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0051c0 0x7f318c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a5e00640 1 -- 172.21.15.204:0/2672757786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c003520 con 0x7f318c0039b0 2024-04-26T21:45:33.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.449+0000 7f31a6800640 1 -- 172.21.15.204:0/2672757786 reap_dead start 2024-04-26T21:45:33.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31a5e00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 0x7f318c003dd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3198009ac0 tx=0x7f318c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196b6aecd0c48b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198036070 con 0x7f318c0039b0 2024-04-26T21:45:33.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3198031400 con 0x7f318c0039b0 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31a4a00640 1 -- 172.21.15.204:0/2672757786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198009690 con 0x7f318c0039b0 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31ace00640 1 -- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 msgr2=0x7f318c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.450+0000 7f31ace00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 0x7f318c003dd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3198009ac0 tx=0x7f318c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.451+0000 7f31ace00640 1 -- 172.21.15.204:0/2672757786 shutdown_connections 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.451+0000 7f31ace00640 1 --2- 172.21.15.204:0/2672757786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0039b0 0x7f318c003dd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.451+0000 7f31ace00640 1 -- 172.21.15.204:0/2672757786 >> 172.21.15.204:0/2672757786 conn(0x7f31a809e4e0 msgr2=0x7f31a80d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.451+0000 7f31ace00640 1 -- 172.21.15.204:0/2672757786 shutdown_connections 2024-04-26T21:45:33.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.451+0000 7f31ace00640 1 -- 172.21.15.204:0/2672757786 wait complete. 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.452+0000 7f31ace00640 1 Processor -- start 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.453+0000 7f31ace00640 1 -- start start 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.453+0000 7f31ace00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0039b0 0x7f31a81545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.453+0000 7f31ace00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31ace00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 0x7f31a80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31ace00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80d3140 con 0x7f318c0039b0 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31ace00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80d0f20 con 0x7f31a80ba9a0 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31ace00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31a80d32c0 con 0x7f31a80ba540 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 0x7f31a80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55378/0 (socket says 172.21.15.204:55378) 2024-04-26T21:45:33.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 -- 172.21.15.204:0/3759334236 learned_addr learned my addr 172.21.15.204:0/3759334236 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 -- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 msgr2=0x7f31a80ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 -- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 msgr2=0x7f31a80ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5400640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a5e00640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0039b0 0x7f31a81545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 -- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 msgr2=0x7f31a80ca490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 -- 172.21.15.204:0/3759334236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0039b0 msgr2=0x7f31a81545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0039b0 0x7f31a81545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 -- 172.21.15.204:0/3759334236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a80c8cd0 con 0x7f31a80ba9a0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.454+0000 7f31a6800640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 0x7f31a80ca9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31a808e630 tx=0x7f31a000e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0019070 con 0x7f31a80ba9a0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0006d60 con 0x7f31a80ba9a0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0003bb0 con 0x7f31a80ba9a0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a8154ae0 con 0x7f31a80ba9a0 2024-04-26T21:45:33.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a80c9270 con 0x7f31a80ba9a0 2024-04-26T21:45:33.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31a001c070 con 0x7f31a80ba9a0 2024-04-26T21:45:33.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.455+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f31a0014640 con 0x7f31a80ba9a0 2024-04-26T21:45:33.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.456+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a80c9d30 con 0x7f31a80ba9a0 2024-04-26T21:45:33.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.459+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f31a80c9d30 con 0x7f31a80ba9a0 2024-04-26T21:45:33.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.538+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2434bd98-00f9-476b-9fc9-1d32817f20a1", "id": 1} v 0) -- 0x7f31a8069650 con 0x7f31a80ba9a0 2024-04-26T21:45:33.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.544+0000 7f319e000640 1 -- 172.21.15.204:0/3759334236 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2434bd98-00f9-476b-9fc9-1d32817f20a1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f31a0023070 con 0x7f31a80ba9a0 2024-04-26T21:45:33.545 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 msgr2=0x7f31a80ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 0x7f31a80ca9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31a808e630 tx=0x7f31a000e9e0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 shutdown_connections 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a80ba9a0 0x7f31a80ca9d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a80ba540 0x7f31a80ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 --2- 172.21.15.204:0/3759334236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c0039b0 0x7f31a81545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 >> 172.21.15.204:0/3759334236 conn(0x7f31a809e4e0 msgr2=0x7f31a80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 shutdown_connections 2024-04-26T21:45:33.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.546+0000 7f31ace00640 1 -- 172.21.15.204:0/3759334236 wait complete. 2024-04-26T21:45:33.563 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 7232f99a-d698-4033-8f17-49f18b27bbdf 2 2024-04-26T21:45:33.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.725+0000 7f4a8ea00640 1 Processor -- start 2024-04-26T21:45:33.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.726+0000 7f4a8ea00640 1 -- start start 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8ea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a880d3c60 0x7f4a8816a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8ea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a880d0170 con 0x7f4a8815f7f0 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8ea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a880d0450 con 0x7f4a880cfb90 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8ea00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4a880d0730 con 0x7f4a880d3c60 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8cc00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f4a880cfb90 0x7f4a880d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43666/0 (socket says 172.21.15.204:43666) 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8cc00640 1 -- 172.21.15.204:0/1945551776 learned_addr learned my addr 172.21.15.204:0/1945551776 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:33.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.727+0000 7f4a8e000640 1 --2- 172.21.15.204:0/1945551776 >> v2:172.21.15.106:3300/0 conn(0x7f4a880d3c60 0x7f4a8816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237712837 0 0) 0x7f4a880d0450 con 0x7f4a880cfb90 2024-04-26T21:45:33.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a70003520 con 0x7f4a880cfb90 2024-04-26T21:45:33.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761117944 0 0) 0x7f4a70003520 con 0x7f4a880cfb90 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 >> v2:172.21.15.106:3300/0 conn(0x7f4a880d3c60 msgr2=0x7f4a8816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 --2- 172.21.15.204:0/1945551776 >> v2:172.21.15.106:3300/0 conn(0x7f4a880d3c60 0x7f4a8816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 >> v1:172.21.15.106:6789/0 conn(0x7f4a8815f7f0 legacy=0x7f4a8815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880d0a10 con 0x7f4a880cfb90 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f4a74001670 con 0x7f4a880cfb90 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 >> v1:172.21.15.204:6789/0 conn(0x7f4a880cfb90 legacy=0x7f4a880d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a70003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70004800 0x7f4a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a700052e0 0x7f4a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880d0730 con 0x7f4a70004800 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7000c1b0 con 0x7f4a70003a70 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.728+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7000c3b0 con 0x7f4a700052e0 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a70003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8e000640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a700052e0 0x7f4a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8cc00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70004800 0x7f4a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 -- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a700052e0 msgr2=0x7f4a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a700052e0 0x7f4a7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 -- 172.21.15.204:0/1945551776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70004800 msgr2=0x7f4a70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70004800 0x7f4a70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8d600640 1 -- 172.21.15.204:0/1945551776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a70003520 con 0x7f4a70003a70 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8e000640 1 -- 172.21.15.204:0/1945551776 reap_dead start 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.729+0000 7f4a8cc00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70004800 0x7f4a70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:33.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.730+0000 7f4a8d600640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a70003e90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c001e50 tx=0x7f4a70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6cca49c2f9b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.730+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c001a70 con 0x7f4a70003a70 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.730+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c003970 con 0x7f4a70003a70 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.730+0000 7f4a87e00640 1 -- 172.21.15.204:0/1945551776 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c003d60 con 0x7f4a70003a70 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.731+0000 7f4a8ea00640 1 -- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 msgr2=0x7f4a70003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.731+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a70003e90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c001e50 tx=0x7f4a70005830 comp rx=0 tx=0).stop 2024-04-26T21:45:33.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.731+0000 7f4a8ea00640 1 -- 172.21.15.204:0/1945551776 shutdown_connections 2024-04-26T21:45:33.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.731+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/1945551776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a70003e90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.731+0000 7f4a8ea00640 1 -- 172.21.15.204:0/1945551776 >> 172.21.15.204:0/1945551776 conn(0x7f4a880d4f80 msgr2=0x7f4a880d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.732+0000 7f4a8ea00640 1 -- 172.21.15.204:0/1945551776 shutdown_connections 2024-04-26T21:45:33.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.732+0000 7f4a8ea00640 1 -- 172.21.15.204:0/1945551776 wait complete. 2024-04-26T21:45:33.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.734+0000 7f4a8ea00640 1 Processor -- start 2024-04-26T21:45:33.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.734+0000 7f4a8ea00640 1 -- start start 2024-04-26T21:45:33.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a880cb230 0x7f4a881f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8815f5b0 con 0x7f4a880cb230 2024-04-26T21:45:33.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8815f430 con 0x7f4a70003a70 2024-04-26T21:45:33.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8815fbd0 con 0x7f4a8814b1c0 2024-04-26T21:45:33.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33116/0 (socket says 172.21.15.204:33116) 2024-04-26T21:45:33.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 -- 172.21.15.204:0/2662752103 learned_addr learned my addr 172.21.15.204:0/2662752103 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:33.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8cc00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8cc00640 1 -- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 msgr2=0x7f4a8815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8cc00640 1 -- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 msgr2=0x7f4a8815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8cc00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8e000640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a880cb230 0x7f4a881f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8cc00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 -- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 msgr2=0x7f4a8815ce80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 -- 172.21.15.204:0/2662752103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a880cb230 msgr2=0x7f4a881f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a880cb230 0x7f4a881f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.736+0000 7f4a8d600640 1 -- 172.21.15.204:0/2662752103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880cbb20 con 0x7f4a70003a70 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a8d600640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c001f70 tx=0x7f4a7c0014a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c03f020 con 0x7f4a70003a70 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c0052d0 con 0x7f4a70003a70 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c037040 con 0x7f4a70003a70 2024-04-26T21:45:33.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a880cd1d0 con 0x7f4a70003a70 2024-04-26T21:45:33.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.737+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a880cd790 con 0x7f4a70003a70 2024-04-26T21:45:33.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.738+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a7c042020 con 0x7f4a70003a70 2024-04-26T21:45:33.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.738+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f4a7c0056a0 con 0x7f4a70003a70 2024-04-26T21:45:33.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.739+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a880d5df0 con 0x7f4a70003a70 2024-04-26T21:45:33.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.741+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4a880d5df0 con 0x7f4a70003a70 2024-04-26T21:45:33.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.832+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7232f99a-d698-4033-8f17-49f18b27bbdf", "id": 2} v 0) -- 0x7f4a8814b5c0 con 0x7f4a70003a70 2024-04-26T21:45:33.833 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:33.832+0000 7fe6ade18040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:45:33.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.840+0000 7f4a85600640 1 -- 172.21.15.204:0/2662752103 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7232f99a-d698-4033-8f17-49f18b27bbdf", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f4a7c049030 con 0x7f4a70003a70 2024-04-26T21:45:33.841 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 msgr2=0x7f4a880c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c001f70 tx=0x7f4a7c0014a0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 shutdown_connections 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a880cb230 0x7f4a881f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8814b1c0 0x7f4a8815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 --2- 172.21.15.204:0/2662752103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a70003a70 0x7f4a880c8fa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 >> 172.21.15.204:0/2662752103 conn(0x7f4a880d4f80 msgr2=0x7f4a880ba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 shutdown_connections 2024-04-26T21:45:33.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:33.842+0000 7f4a8ea00640 1 -- 172.21.15.204:0/2662752103 wait complete. 2024-04-26T21:45:33.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 4dcbdf81-a9a2-4a82-9270-7ef5048f8cae 3 2024-04-26T21:45:33.864 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:33.864+0000 7faf2e91c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:45:33.936 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:33.935+0000 7fe6ade18040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:33.979 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:33.979+0000 7faf2e91c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:33.986 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:33.984+0000 7fe6ade18040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:45:34.036 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:34.035+0000 7faf2e91c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:45:34.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.036+0000 7fc5fd000640 1 Processor -- start 2024-04-26T21:45:34.037 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.036+0000 7fc5fd000640 1 -- start start 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5fd000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc5f80b67e0 0x7fc5f806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5fd000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5f8073c80 con 0x7fc5f806c350 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5fd000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5f8073f60 con 0x7fc5f80b6c60 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5fd000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f80b4ba0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5f7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc5f80b6c60 0x7fc5f806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43674/0 (socket says 172.21.15.204:43674) 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5f7e00640 1 -- 172.21.15.204:0/1370952213 learned_addr learned my addr 172.21.15.204:0/1370952213 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.038+0000 7fc5efe00640 1 --2- 172.21.15.204:0/1370952213 >> v2:172.21.15.106:3300/0 conn(0x7fc5f80b67e0 0x7fc5f806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.039+0000 7fc5efe00640 1 -- 172.21.15.204:0/1370952213 >> v1:172.21.15.204:6789/0 conn(0x7fc5f80b6c60 legacy=0x7fc5f806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.039+0000 7fc5efe00640 1 -- 172.21.15.204:0/1370952213 >> v1:172.21.15.106:6789/0 conn(0x7fc5f806c350 legacy=0x7fc5f806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.039+0000 7fc5efe00640 1 -- 172.21.15.204:0/1370952213 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f80b4e80 con 0x7fc5f80b67e0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.040+0000 7fc5efe00640 1 --2- 172.21.15.204:0/1370952213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f80b67e0 0x7fc5f806f160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00097a0 tx=0x7fc5e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bc15ddf58ff386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.041+0000 7fc5f6a00640 1 -- 172.21.15.204:0/1370952213 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0037070 con 0x7fc5f80b67e0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.041+0000 7fc5f6a00640 1 -- 172.21.15.204:0/1370952213 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e0032440 con 0x7fc5f80b67e0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.041+0000 7fc5f6a00640 1 -- 172.21.15.204:0/1370952213 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0032810 con 0x7fc5f80b67e0 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 -- 172.21.15.204:0/1370952213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f80b67e0 msgr2=0x7fc5f806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 --2- 172.21.15.204:0/1370952213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f80b67e0 0x7fc5f806f160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00097a0 tx=0x7fc5e002a230 comp rx=0 tx=0).stop 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 -- 172.21.15.204:0/1370952213 shutdown_connections 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 --2- 172.21.15.204:0/1370952213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f80b67e0 0x7fc5f806f160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 -- 172.21.15.204:0/1370952213 >> 172.21.15.204:0/1370952213 conn(0x7fc5f80b38a0 msgr2=0x7fc5f80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 -- 172.21.15.204:0/1370952213 shutdown_connections 2024-04-26T21:45:34.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.042+0000 7fc5fd000640 1 -- 172.21.15.204:0/1370952213 wait complete. 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.044+0000 7fc5fd000640 1 Processor -- start 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.044+0000 7fc5fd000640 1 -- start start 2024-04-26T21:45:34.047 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 0x7fc5f80c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f80aea40 con 0x7fc5f806c350 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f80ae8c0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5fd000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f80b4520 con 0x7fc5f80b6c60 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.045+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33132/0 (socket says 172.21.15.204:33132) 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 -- 172.21.15.204:0/180875876 learned_addr learned my addr 172.21.15.204:0/180875876 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7400640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 0x7fc5f80c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7e00640 1 -- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 msgr2=0x7fc5f8068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7e00640 1 -- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 msgr2=0x7fc5f8068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7e00640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7e00640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 -- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 msgr2=0x7fc5f8068300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 -- 172.21.15.204:0/180875876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 msgr2=0x7fc5f80c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 0x7fc5f80c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 -- 172.21.15.204:0/180875876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f80c7a60 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5f7400640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 0x7fc5f80c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.046+0000 7fc5efe00640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5e002a270 tx=0x7fc5e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0037040 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e0042070 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0030bb0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f8147fd0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f8147210 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5e0047460 con 0x7fc5f80b67e0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.047+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fc5e003d8d0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.048+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f8069ad0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.052+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc5f80c7a60 con 0x7fc5f80b67e0 2024-04-26T21:45:34.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.130+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4dcbdf81-a9a2-4a82-9270-7ef5048f8cae", "id": 3} v 0) -- 0x7fc5f806cfc0 con 0x7fc5f80b67e0 2024-04-26T21:45:34.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.138+0000 7fc5ef400640 1 -- 172.21.15.204:0/180875876 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4dcbdf81-a9a2-4a82-9270-7ef5048f8cae", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fc5e003d440 con 0x7fc5f80b67e0 2024-04-26T21:45:34.138 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 msgr2=0x7fc5f80c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5e002a270 tx=0x7fc5e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 shutdown_connections 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5f80b6c60 0x7fc5f8068300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5f80b67e0 0x7fc5f80c7520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 --2- 172.21.15.204:0/180875876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f806c350 0x7fc5f80c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 >> 172.21.15.204:0/180875876 conn(0x7fc5f80b38a0 msgr2=0x7fc5f80af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.139+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 shutdown_connections 2024-04-26T21:45:34.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.140+0000 7fc5fd000640 1 -- 172.21.15.204:0/180875876 wait complete. 2024-04-26T21:45:34.157 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 2cdc59f0-3c29-45d4-a2d9-9a3ba525b456 4 2024-04-26T21:45:34.178 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:34.177+0000 7fe6ade18040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:45:34.233 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:34.232+0000 7fe6ade18040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:45:34.250 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:34.249+0000 7faf2e91c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:45:34.293 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:34.292+0000 7fe6ade18040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:45:34.320 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:34.318+0000 7faf2e91c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:45:34.386 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.384+0000 7f99eae00640 1 Processor -- start 2024-04-26T21:45:34.386 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:34.385+0000 7faf2e91c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:45:34.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.385+0000 7f99eae00640 1 -- start start 2024-04-26T21:45:34.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.386+0000 7f99eae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99e406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99eae00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e4163c70 con 0x7f99e406e590 2024-04-26T21:45:34.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99eae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e40d1370 con 0x7f99e41635b0 2024-04-26T21:45:34.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99eae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99e40d1650 con 0x7f99e40cece0 2024-04-26T21:45:34.389 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99ea400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f99e41635b0 0x7f99e415ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43682/0 (socket says 172.21.15.204:43682) 2024-04-26T21:45:34.389 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99ea400640 1 -- 172.21.15.204:0/2066606894 learned_addr learned my addr 172.21.15.204:0/2066606894 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:34.389 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e9a00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99e406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e9a00640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 msgr2=0x7f99e406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e9a00640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 msgr2=0x7f99e406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e9a00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99e406e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423795884 0 0) 0x7f99e40d1370 con 0x7f99e41635b0 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.387+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99cc003520 con 0x7f99e41635b0 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e9a00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99e406e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755006896 0 0) 0x7f99e4163c70 con 0x7f99e406e590 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e40d1370 con 0x7f99e406e590 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4118284827 0 0) 0x7f99cc003520 con 0x7f99e41635b0 2024-04-26T21:45:34.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 msgr2=0x7f99e406e050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99e406e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 >> v1:172.21.15.106:6789/0 conn(0x7f99e406e590 legacy=0x7f99e4162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e40d1930 con 0x7f99e41635b0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f99dc0019f0 con 0x7f99e41635b0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 >> v1:172.21.15.204:6789/0 conn(0x7f99e41635b0 legacy=0x7f99e415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99cc003bb0 0x7f99cc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99e40d1650 con 0x7f99cc003bb0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc00bb80 con 0x7f99cc004d90 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99cc00bd80 con 0x7f99e40cece0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99ea400640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e9000640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.393 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.388+0000 7f99e9a00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99cc003bb0 0x7f99cc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.389+0000 7f99ea400640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 msgr2=0x7f99cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.389+0000 7f99ea400640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e40cece0 0x7f99cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.389+0000 7f99ea400640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99cc003bb0 msgr2=0x7f99cc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.389+0000 7f99ea400640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99cc003bb0 0x7f99cc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.389+0000 7f99ea400640 1 -- 172.21.15.204:0/2066606894 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc003520 con 0x7f99cc004d90 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.390+0000 7f99ea400640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99cc005210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00a720 tx=0x7f99cc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96603406b27104d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.390+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00a370 con 0x7f99cc004d90 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.390+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99dc02f070 con 0x7f99cc004d90 2024-04-26T21:45:34.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.390+0000 7f99e3e00640 1 -- 172.21.15.204:0/2066606894 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc02bb90 con 0x7f99cc004d90 2024-04-26T21:45:34.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 -- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 msgr2=0x7f99cc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 --2- 172.21.15.204:0/2066606894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99cc005210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00a720 tx=0x7f99cc005830 comp rx=0 tx=0).stop 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99ea400640 1 -- 172.21.15.204:0/2066606894 reap_dead start 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 -- 172.21.15.204:0/2066606894 shutdown_connections 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 -- 172.21.15.204:0/2066606894 >> 172.21.15.204:0/2066606894 conn(0x7f99e40d2de0 msgr2=0x7f99e40d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 -- 172.21.15.204:0/2066606894 shutdown_connections 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.391+0000 7f99eae00640 1 -- 172.21.15.204:0/2066606894 wait complete. 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.393+0000 7f99eae00640 1 Processor -- start 2024-04-26T21:45:34.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.394+0000 7f99eae00640 1 -- start start 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e4150c90 0x7f99e4152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99e40ce440 con 0x7f99e4150c90 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99e40d1ef0 con 0x7f99cc004d90 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99eae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99e40d2410 con 0x7f99e4151110 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99e9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99e9000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e4150c90 0x7f99e4152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99e9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33164/0 (socket says 172.21.15.204:33164) 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99e9a00640 1 -- 172.21.15.204:0/1031084720 learned_addr learned my addr 172.21.15.204:0/1031084720 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.395+0000 7f99ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99ea400640 1 -- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 msgr2=0x7f99e4153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99ea400640 1 -- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 msgr2=0x7f99e4153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99ea400640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99ea400640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 -- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 msgr2=0x7f99e4153040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 -- 172.21.15.204:0/1031084720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e4150c90 msgr2=0x7f99e4152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e4150c90 0x7f99e4152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 -- 172.21.15.204:0/1031084720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e415fa00 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e9a00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99d80108a0 tx=0x7f99d8010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d801a020 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.396+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d800e9c0 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.397+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e414fbf0 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.397+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d800edb0 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.397+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41501b0 con 0x7f99cc004d90 2024-04-26T21:45:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.397+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f99d801d020 con 0x7f99cc004d90 2024-04-26T21:45:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.397+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f99d8016d20 con 0x7f99cc004d90 2024-04-26T21:45:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.398+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4069ad0 con 0x7f99cc004d90 2024-04-26T21:45:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.400+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f99d8024020 con 0x7f99cc004d90 2024-04-26T21:45:34.421 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:34.420+0000 7fe6ade18040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:34.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.476+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2cdc59f0-3c29-45d4-a2d9-9a3ba525b456", "id": 4} v 0) -- 0x7f99e41630c0 con 0x7f99cc004d90 2024-04-26T21:45:34.487 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.485+0000 7f99e1600640 1 -- 172.21.15.204:0/1031084720 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2cdc59f0-3c29-45d4-a2d9-9a3ba525b456", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f99d8016810 con 0x7f99cc004d90 2024-04-26T21:45:34.487 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 msgr2=0x7f99e414f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99d80108a0 tx=0x7f99d8010ca0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 shutdown_connections 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e4151110 0x7f99e4153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e4150c90 0x7f99e4152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 --2- 172.21.15.204:0/1031084720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99cc004d90 0x7f99e414f610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 >> 172.21.15.204:0/1031084720 conn(0x7f99e40d2de0 msgr2=0x7f99e41620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 shutdown_connections 2024-04-26T21:45:34.489 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.487+0000 7f99eae00640 1 -- 172.21.15.204:0/1031084720 wait complete. 2024-04-26T21:45:34.504 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 999a8669-c8b6-4599-811c-4531d20d4c26 5 2024-04-26T21:45:34.527 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:34.526+0000 7faf2e91c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:34.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.673+0000 7f7dbf200640 1 Processor -- start 2024-04-26T21:45:34.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.673+0000 7f7dbf200640 1 -- start start 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.674+0000 7f7dbf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.674+0000 7f7dbf200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00d0170 con 0x7f7dc00d2c50 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.674+0000 7f7dbf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00d0450 con 0x7f7dc015fbd0 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.674+0000 7f7dbf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc00d0730 con 0x7f7dc015f7f0 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55444/0 (socket says 172.21.15.204:55444) 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 learned_addr learned my addr 172.21.15.204:0/3545808325 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbd400640 1 --1- 172.21.15.204:0/3545808325 >> v1:172.21.15.204:6789/0 conn(0x7f7dc015fbd0 0x7f7dc00d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43698/0 (socket says 172.21.15.204:43698) 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 msgr2=0x7f7dc00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:45:34.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 msgr2=0x7f7dc00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021560470 0 0) 0x7f7dc00d0450 con 0x7f7dc015fbd0 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.675+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da4003520 con 0x7f7dc015fbd0 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3381938652 0 0) 0x7f7da4003520 con 0x7f7dc015fbd0 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 msgr2=0x7f7dc00cfb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7dc00cfb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 >> v1:172.21.15.106:6789/0 conn(0x7f7dc00d2c50 legacy=0x7f7dc00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00d0a10 con 0x7f7dc015fbd0 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f7dac001670 con 0x7f7dc015fbd0 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 >> v1:172.21.15.204:6789/0 conn(0x7f7dc015fbd0 legacy=0x7f7dc00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da4003a20 0x7f7da4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7da4004800 0x7f7da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7da400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00d0730 con 0x7f7da4004800 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4005520 con 0x7f7da4003a20 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7da400bf00 con 0x7f7dc015f7f0 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da4003a20 0x7f7da4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbe800640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7da400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbd400640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7da4004800 0x7f7da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 msgr2=0x7f7da400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dc015f7f0 0x7f7da400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7da4004800 msgr2=0x7f7da4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7da4004800 0x7f7da4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.676+0000 7f7dbde00640 1 -- 172.21.15.204:0/3545808325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da4003520 con 0x7f7da4003a20 2024-04-26T21:45:34.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.677+0000 7f7dbde00640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da4003a20 0x7f7da4003e00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009890 tx=0x7f7da4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbccf206c93ed36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0037070 con 0x7f7da4003a20 2024-04-26T21:45:34.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db002ad00 con 0x7f7da4003a20 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbca00640 1 -- 172.21.15.204:0/3545808325 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db000f040 con 0x7f7da4003a20 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbf200640 1 -- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da4003a20 msgr2=0x7f7da4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbf200640 1 --2- 172.21.15.204:0/3545808325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da4003a20 0x7f7da4003e00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009890 tx=0x7f7da4005830 comp rx=0 tx=0).stop 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbe800640 1 -- 172.21.15.204:0/3545808325 reap_dead start 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbf200640 1 -- 172.21.15.204:0/3545808325 shutdown_connections 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.678+0000 7f7dbf200640 1 -- 172.21.15.204:0/3545808325 >> 172.21.15.204:0/3545808325 conn(0x7f7dc00d4f80 msgr2=0x7f7dc00d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.679+0000 7f7dbf200640 1 -- 172.21.15.204:0/3545808325 shutdown_connections 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.679+0000 7f7dbf200640 1 -- 172.21.15.204:0/3545808325 wait complete. 2024-04-26T21:45:34.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.681+0000 7f7dbf200640 1 Processor -- start 2024-04-26T21:45:34.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.681+0000 7f7dbf200640 1 -- start start 2024-04-26T21:45:34.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 0x7f7dc0159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 0x7f7dc00ba030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc015f430 con 0x7f7dc00ce9e0 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0160050 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc015f5b0 con 0x7f7da4003a20 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbe800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 0x7f7dc00ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55454/0 (socket says 172.21.15.204:55454) 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 -- 172.21.15.204:0/1259800079 learned_addr learned my addr 172.21.15.204:0/1259800079 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbd400640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 0x7f7dc0159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 -- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 msgr2=0x7f7dc014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 -- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 msgr2=0x7f7dc014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:34.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.683+0000 7f7dbde00640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 -- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 msgr2=0x7f7dc014d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 -- 172.21.15.204:0/1259800079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 msgr2=0x7f7dc0159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 0x7f7dc0159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 -- 172.21.15.204:0/1259800079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc015a510 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbd400640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 0x7f7dc0159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbe800640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 0x7f7dc00ba030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7dc008f560 tx=0x7f7db800ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8003b80 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8018070 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc015a7e0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db80050d0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.684+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc015ada0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.685+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7db8005270 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.685+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7db8005560 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.686+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc00ba570 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.688+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7db800d6e0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.768 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.766+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "999a8669-c8b6-4599-811c-4531d20d4c26", "id": 5} v 0) -- 0x7f7dc00693c0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.776+0000 7f7db6000640 1 -- 172.21.15.204:0/1259800079 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "999a8669-c8b6-4599-811c-4531d20d4c26", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7db800d8c0 con 0x7f7dc00b9bb0 2024-04-26T21:45:34.778 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 msgr2=0x7f7dc00ba030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 0x7f7dc00ba030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7dc008f560 tx=0x7f7db800ff50 comp rx=0 tx=0).stop 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 shutdown_connections 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dc00b9bb0 0x7f7dc00ba030 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7dc00ce9e0 0x7f7dc0159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 --2- 172.21.15.204:0/1259800079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7da4003a20 0x7f7dc014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 >> 172.21.15.204:0/1259800079 conn(0x7f7dc00d4f80 msgr2=0x7f7dc00d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 shutdown_connections 2024-04-26T21:45:34.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:34.778+0000 7f7dbf200640 1 -- 172.21.15.204:0/1259800079 wait complete. 2024-04-26T21:45:34.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f1bc9749-7a5c-4612-9b9b-485f6a8fdeef 6 2024-04-26T21:45:34.873 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:34.872+0000 7fe6ade18040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:45:35.024 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.023+0000 7faf2e91c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:45:35.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.038+0000 7f41a8200640 1 Processor -- start 2024-04-26T21:45:35.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.038+0000 7f41a8200640 1 -- start start 2024-04-26T21:45:35.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a8200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41a00d58b0 0x7f41a00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a8200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a00d0960 con 0x7f41a00d3780 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a8200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f41a00b2070 con 0x7f41a00d58b0 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00d0c40 con 0x7f41a00b2a40 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a5800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f41a00d3780 0x7f41a00d3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43704/0 (socket says 172.21.15.204:43704) 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a5800640 1 -- 172.21.15.204:0/1651879767 learned_addr learned my addr 172.21.15.204:0/1651879767 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.039+0000 7f41a6200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a4e00640 1 --2- 172.21.15.204:0/1651879767 >> v2:172.21.15.204:3300/0 conn(0x7f41a00d58b0 0x7f41a00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a6200640 1 -- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 msgr2=0x7f41a00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a6200640 1 -- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 msgr2=0x7f41a00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a6200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a6200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f419fe00640 1 -- 172.21.15.204:0/1651879767 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 835944299 0 0) 0x7f41a00d0960 con 0x7f41a00d3780 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f419fe00640 1 -- 172.21.15.204:0/1651879767 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4188003520 con 0x7f41a00d3780 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a4e00640 1 -- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 msgr2=0x7f41a00b2ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a4e00640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a4e00640 1 -- 172.21.15.204:0/1651879767 >> v1:172.21.15.204:6789/0 conn(0x7f41a00d3780 legacy=0x7f41a00d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.040+0000 7f41a4e00640 1 -- 172.21.15.204:0/1651879767 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00d0f20 con 0x7f41a00d58b0 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.041+0000 7f41a4e00640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00b2500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f418c0097a0 tx=0x7f418c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d8302f592b56f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.041+0000 7f419fe00640 1 -- 172.21.15.204:0/1651879767 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c037070 con 0x7f41a00d58b0 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.041+0000 7f419fe00640 1 -- 172.21.15.204:0/1651879767 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c032440 con 0x7f41a00d58b0 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.041+0000 7f419fe00640 1 -- 172.21.15.204:0/1651879767 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c032810 con 0x7f41a00d58b0 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 -- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 msgr2=0x7f41a00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00b2500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f418c0097a0 tx=0x7f418c02a180 comp rx=0 tx=0).stop 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 -- 172.21.15.204:0/1651879767 shutdown_connections 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00b2a40 0x7f41a00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 --2- 172.21.15.204:0/1651879767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00b2500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 -- 172.21.15.204:0/1651879767 >> 172.21.15.204:0/1651879767 conn(0x7f41a009e4e0 msgr2=0x7f41a00d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 -- 172.21.15.204:0/1651879767 shutdown_connections 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.042+0000 7f41a8200640 1 -- 172.21.15.204:0/1651879767 wait complete. 2024-04-26T21:45:35.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.044+0000 7f41a8200640 1 Processor -- start 2024-04-26T21:45:35.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.044+0000 7f41a8200640 1 -- start start 2024-04-26T21:45:35.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b2a40 0x7f41a00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41a015f9f0 con 0x7f41a00b2a40 2024-04-26T21:45:35.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41a015f870 con 0x7f41a00d58b0 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00d5d10 con 0x7f41a00d3780 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55466/0 (socket says 172.21.15.204:55466) 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 -- 172.21.15.204:0/606136207 learned_addr learned my addr 172.21.15.204:0/606136207 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a6200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a5800640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b2a40 0x7f41a00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 -- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 msgr2=0x7f41a00b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 -- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 msgr2=0x7f41a00b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.046+0000 7f41a4e00640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 -- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 msgr2=0x7f41a00b7dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 -- 172.21.15.204:0/606136207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b2a40 msgr2=0x7f41a00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b2a40 0x7f41a00ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 -- 172.21.15.204:0/606136207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00b8300 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a6200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00cd8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4198007d20 tx=0x7f419800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419800edd0 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4198003990 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198003dc0 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.047+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a00ba320 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.048+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a00ba8e0 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.048+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f419801b020 con 0x7f41a00d58b0 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.048+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4198014420 con 0x7f41a00d58b0 2024-04-26T21:45:35.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.049+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a0069650 con 0x7f41a00d58b0 2024-04-26T21:45:35.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.051+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f41980149c0 con 0x7f41a00d58b0 2024-04-26T21:45:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.129+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f1bc9749-7a5c-4612-9b9b-485f6a8fdeef", "id": 6} v 0) -- 0x7f41a00d2700 con 0x7f41a00d58b0 2024-04-26T21:45:35.142 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.140+0000 7f419d600640 1 -- 172.21.15.204:0/606136207 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1bc9749-7a5c-4612-9b9b-485f6a8fdeef", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4198014ba0 con 0x7f41a00d58b0 2024-04-26T21:45:35.142 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 msgr2=0x7f41a00cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00cd8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4198007d20 tx=0x7f419800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 shutdown_connections 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41a00d58b0 0x7f41a00cd8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41a00d3780 0x7f41a00b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 --2- 172.21.15.204:0/606136207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b2a40 0x7f41a00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 >> 172.21.15.204:0/606136207 conn(0x7f41a009e4e0 msgr2=0x7f41a00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 shutdown_connections 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.142+0000 7f41a8200640 1 -- 172.21.15.204:0/606136207 wait complete. 2024-04-26T21:45:35.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f32beea4-f84e-49d6-9485-5d68f547e530 7 2024-04-26T21:45:35.165 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:35.164+0000 7fe6ade18040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:45:35.226 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:35.225+0000 7fe6ade18040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:45:35.338 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.337+0000 7faf2e91c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:45:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.341+0000 7f51cb000640 1 Processor -- start 2024-04-26T21:45:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.341+0000 7f51cb000640 1 -- start start 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51cb000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f51c415f1c0 0x7f51c40ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51cb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51cb000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c415f5a0 con 0x7f51c40d2870 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51cb000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f51c40cf950 con 0x7f51c415f1c0 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51cb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f51c40cff10 con 0x7f51c406c350 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51ca600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51c9200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f51c40d2870 0x7f51c40d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43718/0 (socket says 172.21.15.204:43718) 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.342+0000 7f51c9200640 1 -- 172.21.15.204:0/106754324 learned_addr learned my addr 172.21.15.204:0/106754324 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51ca600640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> v2:172.21.15.106:3300/0 conn(0x7f51c415f1c0 0x7f51c40ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51ca600640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51ca600640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c406c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51ca600640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c406c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088588044 0 0) 0x7f51c415f5a0 con 0x7f51c40d2870 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51ac003520 con 0x7f51c40d2870 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2414300632 0 0) 0x7f51ac003520 con 0x7f51c40d2870 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c406c780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c406c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 >> v2:172.21.15.106:3300/0 conn(0x7f51c415f1c0 msgr2=0x7f51c40ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 --2- 172.21.15.204:0/106754324 >> v2:172.21.15.106:3300/0 conn(0x7f51c415f1c0 0x7f51c40ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c4160260 con 0x7f51c40d2870 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f51b0001670 con 0x7f51c40d2870 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 >> v1:172.21.15.204:6789/0 conn(0x7f51c40d2870 legacy=0x7f51c40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51ac003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51ac003fa0 0x7f51ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40cf950 con 0x7f51ac003fa0 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40cff10 con 0x7f51ac004d40 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.343+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac00bb20 con 0x7f51c406c350 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> v2:172.21.15.106:3300/0 conn(0x7f51c415f1c0 0x7f51c40ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51ca600640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51ac003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9200640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51ac003fa0 0x7f51ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51ac004d40 msgr2=0x7f51ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51ac003fa0 msgr2=0x7f51ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51ac003fa0 0x7f51ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51c9c00640 1 -- 172.21.15.204:0/106754324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac003520 con 0x7f51c406c350 2024-04-26T21:45:35.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.344+0000 7f51ca600640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.345+0000 7f51c9c00640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51ac003a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009c70 tx=0x7f51ac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c719f6e1d6e62be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.345+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8009840 con 0x7f51c406c350 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.345+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b8004020 con 0x7f51c406c350 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.345+0000 7f51c3e00640 1 -- 172.21.15.204:0/106754324 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b80043d0 con 0x7f51c406c350 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 -- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51ac003a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 --2- 172.21.15.204:0/106754324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51ac003a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009c70 tx=0x7f51ac005830 comp rx=0 tx=0).stop 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51ca600640 1 -- 172.21.15.204:0/106754324 reap_dead start 2024-04-26T21:45:35.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 -- 172.21.15.204:0/106754324 shutdown_connections 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 -- 172.21.15.204:0/106754324 >> 172.21.15.204:0/106754324 conn(0x7f51c40d1d80 msgr2=0x7f51c40d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 -- 172.21.15.204:0/106754324 shutdown_connections 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.346+0000 7f51cb000640 1 -- 172.21.15.204:0/106754324 wait complete. 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.348+0000 7f51cb000640 1 Processor -- start 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.348+0000 7f51cb000640 1 -- start start 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 0x7f51c41f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40cfba0 con 0x7f51c4070210 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40d2d00 con 0x7f51c41f6380 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51cb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f51c406c780 con 0x7f51c406c350 2024-04-26T21:45:35.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55492/0 (socket says 172.21.15.204:55492) 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33220/0 (socket says 172.21.15.204:33220) 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 -- 172.21.15.204:0/3872468051 learned_addr learned my addr 172.21.15.204:0/3872468051 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51ca600640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 0x7f51c41f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 -- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c40b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 -- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c40b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.349+0000 7f51c9c00640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 -- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 msgr2=0x7f51c40b7050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 -- 172.21.15.204:0/3872468051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 msgr2=0x7f51c41f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 0x7f51c41f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 -- 172.21.15.204:0/3872468051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c4161100 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51ca600640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 0x7f51c41f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c9200640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51b0004f70 tx=0x7f51b00074f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b000f750 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b0018070 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c41613e0 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b000fb10 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.350+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c4160df0 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.351+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f51b000fcb0 con 0x7f51c41f6380 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.351+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f51b001dac0 con 0x7f51c41f6380 2024-04-26T21:45:35.356 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.351+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c4161900 con 0x7f51c41f6380 2024-04-26T21:45:35.356 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.354+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f51b001d460 con 0x7f51c41f6380 2024-04-26T21:45:35.357 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:35.356+0000 7fe6ade18040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:45:35.404 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.403+0000 7faf2e91c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:45:35.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.432+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f32beea4-f84e-49d6-9485-5d68f547e530", "id": 7} v 0) -- 0x7f51c4069ad0 con 0x7f51c41f6380 2024-04-26T21:45:35.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.443+0000 7f51c1600640 1 -- 172.21.15.204:0/3872468051 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f32beea4-f84e-49d6-9485-5d68f547e530", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f51b001d710 con 0x7f51c41f6380 2024-04-26T21:45:35.445 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 msgr2=0x7f51c41608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51b0004f70 tx=0x7f51b00074f0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 shutdown_connections 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51c4070210 0x7f51c41f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c41f6380 0x7f51c41608b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 --2- 172.21.15.204:0/3872468051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51c406c350 0x7f51c40b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 >> 172.21.15.204:0/3872468051 conn(0x7f51c40d1d80 msgr2=0x7f51c40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 shutdown_connections 2024-04-26T21:45:35.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:45:35.445+0000 7f51cb000640 1 -- 172.21.15.204:0/3872468051 wait complete. 2024-04-26T21:45:35.463 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T21:45:35.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T21:45:35.471 INFO:tasks.ceph.osd.0:Started 2024-04-26T21:45:35.471 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T21:45:35.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T21:45:35.478 INFO:tasks.ceph.osd.1:Started 2024-04-26T21:45:35.478 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T21:45:35.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T21:45:35.484 INFO:tasks.ceph.osd.2:Started 2024-04-26T21:45:35.484 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T21:45:35.484 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T21:45:35.489 INFO:tasks.ceph.osd.3:Started 2024-04-26T21:45:35.490 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T21:45:35.490 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T21:45:35.495 INFO:tasks.ceph.osd.4:Started 2024-04-26T21:45:35.496 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T21:45:35.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T21:45:35.502 INFO:tasks.ceph.osd.5:Started 2024-04-26T21:45:35.503 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T21:45:35.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T21:45:35.510 INFO:tasks.ceph.osd.6:Started 2024-04-26T21:45:35.510 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T21:45:35.510 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T21:45:35.517 INFO:tasks.ceph.osd.7:Started 2024-04-26T21:45:35.517 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:35.556 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.555+0000 7faf2e91c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:45:35.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.727+0000 7f6dcca00640 1 Processor -- start 2024-04-26T21:45:35.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.727+0000 7f6dcca00640 1 -- start start 2024-04-26T21:45:35.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.728+0000 7f6dcca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6dc809c160 0x7f6dc81681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.728+0000 7f6dcca00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc8070800 con 0x7f6dc8160660 2024-04-26T21:45:35.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.728+0000 7f6dcca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc8070ae0 con 0x7f6dc80718a0 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.728+0000 7f6dcca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc8070dc0 con 0x7f6dc809c160 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc5e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6dc8160660 0x7f6dc8071dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44794/0 (socket says 172.21.15.106:44794) 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc5e00640 1 -- 172.21.15.106:0/4021913866 learned_addr learned my addr 172.21.15.106:0/4021913866 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc6800640 1 --2- 172.21.15.106:0/4021913866 >> v2:172.21.15.204:3300/0 conn(0x7f6dc809c160 0x7f6dc81681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc5400640 1 -- 172.21.15.106:0/4021913866 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891149876 0 0) 0x7f6dc8070800 con 0x7f6dc8160660 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc5400640 1 -- 172.21.15.106:0/4021913866 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dac003520 con 0x7f6dc8160660 2024-04-26T21:45:35.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc6800640 1 -- 172.21.15.106:0/4021913866 >> v1:172.21.15.204:6789/0 conn(0x7f6dc80718a0 legacy=0x7f6dc809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc6800640 1 -- 172.21.15.106:0/4021913866 >> v1:172.21.15.106:6789/0 conn(0x7f6dc8160660 legacy=0x7f6dc8071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.729+0000 7f6dc6800640 1 -- 172.21.15.106:0/4021913866 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc80710a0 con 0x7f6dc809c160 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.730+0000 7f6dc6800640 1 --2- 172.21.15.106:0/4021913866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc809c160 0x7f6dc81681b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009dc0 tx=0x7f6dc002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc2944e0c252580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.730+0000 7f6dc5400640 1 -- 172.21.15.106:0/4021913866 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0007740 con 0x7f6dc809c160 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.730+0000 7f6dc5400640 1 -- 172.21.15.106:0/4021913866 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc00078e0 con 0x7f6dc809c160 2024-04-26T21:45:35.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.730+0000 7f6dc5400640 1 -- 172.21.15.106:0/4021913866 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0007cb0 con 0x7f6dc809c160 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 -- 172.21.15.106:0/4021913866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc809c160 msgr2=0x7f6dc81681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 --2- 172.21.15.106:0/4021913866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc809c160 0x7f6dc81681b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009dc0 tx=0x7f6dc002a730 comp rx=0 tx=0).stop 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 -- 172.21.15.106:0/4021913866 shutdown_connections 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 --2- 172.21.15.106:0/4021913866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc809c160 0x7f6dc81681b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 -- 172.21.15.106:0/4021913866 >> 172.21.15.106:0/4021913866 conn(0x7f6dc80b4b70 msgr2=0x7f6dc80b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 -- 172.21.15.106:0/4021913866 shutdown_connections 2024-04-26T21:45:35.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.731+0000 7f6dcca00640 1 -- 172.21.15.106:0/4021913866 wait complete. 2024-04-26T21:45:35.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.733+0000 7f6dcca00640 1 Processor -- start 2024-04-26T21:45:35.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.733+0000 7f6dcca00640 1 -- start start 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 0x7f6dc80ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8070420 con 0x7f6dc80718a0 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80724e0 con 0x7f6dc8160660 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dcca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80705a0 con 0x7f6dc809c160 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53850/0 (socket says 172.21.15.106:53850) 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 -- 172.21.15.106:0/3252920436 learned_addr learned my addr 172.21.15.106:0/3252920436 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dbfe00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc6800640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 0x7f6dc80ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dbfe00640 1 -- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 msgr2=0x7f6dc80b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dbfe00640 1 -- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 msgr2=0x7f6dc80b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dbfe00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dbfe00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 -- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 msgr2=0x7f6dc80b7eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 -- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 msgr2=0x7f6dc80ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 0x7f6dc80ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc5e00640 1 -- 172.21.15.106:0/3252920436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc80b85b0 con 0x7f6dc80718a0 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.734+0000 7f6dc6800640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 0x7f6dc80ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dc5e00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c410 tx=0x7f6db800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8017070 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8005ce0 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db80072f0 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc80b8890 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81f28e0 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6db801a070 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.735+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6db8007490 con 0x7f6dc80718a0 2024-04-26T21:45:35.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.736+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc8069540 con 0x7f6dc80718a0 2024-04-26T21:45:35.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.739+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6db8021020 con 0x7f6dc80718a0 2024-04-26T21:45:35.774 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:35.773+0000 7fe6ade18040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:45:35.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.803+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dc809cad0 con 0x7f6dc80718a0 2024-04-26T21:45:35.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.803+0000 7f6dbe000640 1 -- 172.21.15.106:0/3252920436 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6db801c440 con 0x7f6dc80718a0 2024-04-26T21:45:35.805 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:35.805 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:35.436314+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:35.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 msgr2=0x7f6dc81f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c410 tx=0x7f6db800c8d0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 shutdown_connections 2024-04-26T21:45:35.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc8160660 0x7f6dc80ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dc809c160 0x7f6dc80b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 --2- 172.21.15.106:0/3252920436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc80718a0 0x7f6dc81f3430 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 >> 172.21.15.106:0/3252920436 conn(0x7f6dc80b4b70 msgr2=0x7f6dc80b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 shutdown_connections 2024-04-26T21:45:35.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:35.805+0000 7f6dcca00640 1 -- 172.21.15.106:0/3252920436 wait complete. 2024-04-26T21:45:35.822 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T21:45:35.822 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T21:45:35.832 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:35.831+0000 7fe6ade18040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:45:35.925 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.923+0000 7faf2e91c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:45:35.982 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:35.981+0000 7faf2e91c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:45:36.015 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.014+0000 7fe6ade18040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:45:36.167 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:36.165+0000 7faf2e91c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:45:36.171 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-26T21:45:36.169+0000 7fa993c0c640 -1 Falling back to public interface 2024-04-26T21:45:36.174 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-26T21:45:36.174+0000 7efff0bd8640 -1 Falling back to public interface 2024-04-26T21:45:36.175 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-26T21:45:36.174+0000 7f352e415640 -1 Falling back to public interface 2024-04-26T21:45:36.183 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-26T21:45:36.183+0000 7ff9a7333640 -1 Falling back to public interface 2024-04-26T21:45:36.184 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.183+0000 7fe6ade18040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:45:36.202 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-26T21:45:36.200+0000 7f999a03f640 -1 Falling back to public interface 2024-04-26T21:45:36.213 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-26T21:45:36.212+0000 7fec7b1cb640 -1 Falling back to public interface 2024-04-26T21:45:36.214 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-26T21:45:36.213+0000 7fa3c6c0f640 -1 Falling back to public interface 2024-04-26T21:45:36.219 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-26T21:45:36.218+0000 7f5a2480d640 -1 Falling back to public interface 2024-04-26T21:45:36.246 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.246+0000 7fe6ade18040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:45:36.328 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.327+0000 7fe6ade18040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:36.341 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:36.339+0000 7faf2e91c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:45:36.380 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.379+0000 7fe6ade18040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:45:36.400 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:36.398+0000 7faf2e91c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:45:36.483 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:36.481+0000 7faf2e91c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:36.538 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:36.537+0000 7faf2e91c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:45:36.628 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:36.785 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.784+0000 7f3e41c00640 1 Processor -- start 2024-04-26T21:45:36.785 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.784+0000 7f3e41c00640 1 -- start start 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e41c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e3c0d3670 0x7f3e3c0d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e41c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3e3c0d2e70 0x7f3e3c0d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e41c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0b45f0 con 0x7f3e3c0d1730 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e41c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c0d06b0 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e41c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c0b48f0 con 0x7f3e3c0d3670 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e3a000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3e3c0d2e70 0x7f3e3c0d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e3aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e3c0d3670 0x7f3e3c0d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e3b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3e3c0d1730 0x7f3e3c0d1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44804/0 (socket says 172.21.15.106:44804) 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.785+0000 7f3e3b400640 1 -- 172.21.15.106:0/756383000 learned_addr learned my addr 172.21.15.106:0/756383000 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.786+0000 7f3e3a000640 1 -- 172.21.15.106:0/756383000 >> v2:172.21.15.204:3300/0 conn(0x7f3e3c0d3670 msgr2=0x7f3e3c0d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.786+0000 7f3e3a000640 1 --2- 172.21.15.106:0/756383000 >> v2:172.21.15.204:3300/0 conn(0x7f3e3c0d3670 0x7f3e3c0d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.787 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.786+0000 7f3e3a000640 1 -- 172.21.15.106:0/756383000 >> v1:172.21.15.106:6789/0 conn(0x7f3e3c0d1730 legacy=0x7f3e3c0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.788 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.786+0000 7f3e3a000640 1 -- 172.21.15.106:0/756383000 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c0d3f90 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.788 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.787+0000 7f3e3a000640 1 --2- 172.21.15.106:0/756383000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d2e70 0x7f3e3c0d3a50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b7bd0 tx=0x7f3e2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01a4e985dd74f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:36.788 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.787+0000 7f3e39600640 1 -- 172.21.15.106:0/756383000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28037070 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.788 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.787+0000 7f3e39600640 1 -- 172.21.15.106:0/756383000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28032440 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.788 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.787+0000 7f3e39600640 1 -- 172.21.15.106:0/756383000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28032810 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.789 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 -- 172.21.15.106:0/756383000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d2e70 msgr2=0x7f3e3c0d3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.789 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 --2- 172.21.15.106:0/756383000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d2e70 0x7f3e3c0d3a50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b7bd0 tx=0x7f3e2802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.789 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 -- 172.21.15.106:0/756383000 shutdown_connections 2024-04-26T21:45:36.789 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 --2- 172.21.15.106:0/756383000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d2e70 0x7f3e3c0d3a50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.789 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 --2- 172.21.15.106:0/756383000 >> v2:172.21.15.204:3300/0 conn(0x7f3e3c0d3670 0x7f3e3c0d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.790 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 -- 172.21.15.106:0/756383000 >> 172.21.15.106:0/756383000 conn(0x7f3e3c0d5090 msgr2=0x7f3e3c0d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:36.790 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 -- 172.21.15.106:0/756383000 shutdown_connections 2024-04-26T21:45:36.790 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.788+0000 7f3e41c00640 1 -- 172.21.15.106:0/756383000 wait complete. 2024-04-26T21:45:36.790 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.790+0000 7f3e41c00640 1 Processor -- start 2024-04-26T21:45:36.790 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.790+0000 7f3e41c00640 1 -- start start 2024-04-26T21:45:36.793 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.793 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.793 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e3c0d3670 0x7f3e3c0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.793 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0d0ec0 con 0x7f3e3c0d1730 2024-04-26T21:45:36.793 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0d0be0 con 0x7f3e3c0d3670 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e41c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c15f760 con 0x7f3e3c0d2e70 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53868/0 (socket says 172.21.15.106:53868) 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 -- 172.21.15.106:0/3473335590 learned_addr learned my addr 172.21.15.106:0/3473335590 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3a000640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3b400640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e3c0d3670 0x7f3e3c0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3a000640 1 -- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 msgr2=0x7f3e3c0b96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3a000640 1 -- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 msgr2=0x7f3e3c0b96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3a000640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3a000640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 -- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 msgr2=0x7f3e3c0b96a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.794 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 -- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e3c0d3670 msgr2=0x7f3e3c0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e3c0d3670 0x7f3e3c0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.791+0000 7f3e3aa00640 1 -- 172.21.15.106:0/3473335590 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c0ba1b0 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e3aa00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9b0 tx=0x7f3e2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c017070 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c00c9a0 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00cd90 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c0ba490 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1f5720 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e2c01a070 con 0x7f3e3c0d1730 2024-04-26T21:45:36.795 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.792+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3e2c013b40 con 0x7f3e3c0d1730 2024-04-26T21:45:36.796 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.793+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c069e70 con 0x7f3e3c0d1730 2024-04-26T21:45:36.798 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.797+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3e2c013470 con 0x7f3e3c0d1730 2024-04-26T21:45:36.865 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.864+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e3c0d3450 con 0x7f3e3c0d1730 2024-04-26T21:45:36.865 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.865+0000 7f3e32a00640 1 -- 172.21.15.106:0/3473335590 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3e2c013650 con 0x7f3e3c0d1730 2024-04-26T21:45:36.866 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-26T21:45:36.866 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":10,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:35.436314+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:36.867 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 msgr2=0x7f3e3c1f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9b0 tx=0x7f3e2c00ee70 comp rx=0 tx=0).stop 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 shutdown_connections 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e3c0d3670 0x7f3e3c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e3c0d2e70 0x7f3e3c0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 --2- 172.21.15.106:0/3473335590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e3c0d1730 0x7f3e3c1f4a10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.866+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 >> 172.21.15.106:0/3473335590 conn(0x7f3e3c0d5090 msgr2=0x7f3e3c06db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.867+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 shutdown_connections 2024-04-26T21:45:36.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:36.867+0000 7f3e41c00640 1 -- 172.21.15.106:0/3473335590 wait complete. 2024-04-26T21:45:36.880 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:45:36.934 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:36.933+0000 7fe6ade18040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:45:37.066 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:37.065+0000 7fe6ade18040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:45:37.095 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:37.093+0000 7faf2e91c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:45:37.233 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-26T21:45:37.231+0000 7faf2e91c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:45:38.104 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:38.103+0000 7fe64d800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:45:38.107 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:45:38.106+0000 7fe64d800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:45:39.349 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-26T21:45:39.347+0000 7f999a03f640 -1 osd.7 0 log_to_monitors true 2024-04-26T21:45:39.566 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-26T21:45:39.565+0000 7ff9a7333640 -1 osd.3 0 log_to_monitors true 2024-04-26T21:45:39.770 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-26T21:45:39.769+0000 7fa993c0c640 -1 osd.4 0 log_to_monitors true 2024-04-26T21:45:39.828 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-26T21:45:39.827+0000 7f5a2480d640 -1 osd.6 0 log_to_monitors true 2024-04-26T21:45:39.834 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-26T21:45:39.833+0000 7efff0bd8640 -1 osd.0 0 log_to_monitors true 2024-04-26T21:45:39.835 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-26T21:45:39.834+0000 7fa3c6c0f640 -1 osd.5 0 log_to_monitors true 2024-04-26T21:45:39.863 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-26T21:45:39.862+0000 7fec7b1cb640 -1 osd.2 0 log_to_monitors true 2024-04-26T21:45:40.005 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-26T21:45:40.004+0000 7f352e415640 -1 osd.1 0 log_to_monitors true 2024-04-26T21:45:43.443 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-26T21:45:43.442+0000 7f3528c00640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:45:43.449 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-26T21:45:43.448+0000 7f351ec00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.459 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-26T21:45:43.457+0000 7fa98e400640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:45:43.467 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-26T21:45:43.465+0000 7fa984400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.469 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-26T21:45:43.469+0000 7effeba00640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:45:43.477 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-26T21:45:43.476+0000 7effe1a00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.491 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-26T21:45:43.490+0000 7fa3c2000640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:45:43.497 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-26T21:45:43.496+0000 7f9995800640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:45:43.499 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-26T21:45:43.499+0000 7fec76000640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:45:43.500 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-26T21:45:43.498+0000 7fa3b7600640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.502 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-26T21:45:43.501+0000 7f998ae00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.510 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-26T21:45:43.509+0000 7ff9a2c00640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:45:43.513 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-26T21:45:43.512+0000 7fec6c000640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.515 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-26T21:45:43.514+0000 7f5a1f200640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:45:43.517 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-26T21:45:43.516+0000 7ff998200640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.521 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-26T21:45:43.520+0000 7f5a15200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:43.687 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:43.866 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.865+0000 7efcd1e00640 1 Processor -- start 2024-04-26T21:45:43.866 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.865+0000 7efcd1e00640 1 -- start start 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.866+0000 7efcd1e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efccc15f260 0x7efccc06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.866+0000 7efcd1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efccc0b5910 0x7efccc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.866+0000 7efcd1e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efccc072ce0 con 0x7efccc06e510 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efcd1e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efccc06e980 con 0x7efccc15f260 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efcd1e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efccc073580 con 0x7efccc0b5910 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efccc15f260 0x7efccc06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efccc15f260 0x7efccc06fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52338/0 (socket says 172.21.15.106:52338) 2024-04-26T21:45:43.868 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 -- 172.21.15.106:0/2562339571 learned_addr learned my addr 172.21.15.106:0/2562339571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 -- 172.21.15.106:0/2562339571 >> v2:172.21.15.204:3300/0 conn(0x7efccc0b5910 msgr2=0x7efccc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efcca000640 1 --2- 172.21.15.106:0/2562339571 >> v2:172.21.15.204:3300/0 conn(0x7efccc0b5910 0x7efccc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 --2- 172.21.15.106:0/2562339571 >> v2:172.21.15.204:3300/0 conn(0x7efccc0b5910 0x7efccc06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 -- 172.21.15.106:0/2562339571 >> v1:172.21.15.204:6789/0 conn(0x7efccc06e510 legacy=0x7efccc0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.867+0000 7efccaa00640 1 -- 172.21.15.106:0/2562339571 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc072fc0 con 0x7efccc15f260 2024-04-26T21:45:43.869 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.868+0000 7efccaa00640 1 --2- 172.21.15.106:0/2562339571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc15f260 0x7efccc06fec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efccc09f0c0 tx=0x7efcbc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e61c036ada4b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcc9600640 1 -- 172.21.15.106:0/2562339571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc037070 con 0x7efccc15f260 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcc9600640 1 -- 172.21.15.106:0/2562339571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc02ae20 con 0x7efccc15f260 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcc9600640 1 -- 172.21.15.106:0/2562339571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc032660 con 0x7efccc15f260 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 -- 172.21.15.106:0/2562339571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc15f260 msgr2=0x7efccc06fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 --2- 172.21.15.106:0/2562339571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc15f260 0x7efccc06fec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efccc09f0c0 tx=0x7efcbc02a110 comp rx=0 tx=0).stop 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 -- 172.21.15.106:0/2562339571 shutdown_connections 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 --2- 172.21.15.106:0/2562339571 >> v2:172.21.15.204:3300/0 conn(0x7efccc0b5910 0x7efccc06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 --2- 172.21.15.106:0/2562339571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc15f260 0x7efccc06fec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 -- 172.21.15.106:0/2562339571 >> 172.21.15.106:0/2562339571 conn(0x7efccc0d0c40 msgr2=0x7efccc0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.869+0000 7efcd1e00640 1 -- 172.21.15.106:0/2562339571 shutdown_connections 2024-04-26T21:45:43.871 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.870+0000 7efcd1e00640 1 -- 172.21.15.106:0/2562339571 wait complete. 2024-04-26T21:45:43.872 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.871+0000 7efcd1e00640 1 Processor -- start 2024-04-26T21:45:43.872 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.872+0000 7efcd1e00640 1 -- start start 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efccc06e510 0x7efccc14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 0x7efccc152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0b5d70 con 0x7efccc0b5910 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0b3d70 con 0x7efccc15f260 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcd1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efccc0b4910 con 0x7efccc06e510 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcca000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcca000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52350/0 (socket says 172.21.15.106:52350) 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcca000640 1 -- 172.21.15.106:0/1732760919 learned_addr learned my addr 172.21.15.106:0/1732760919 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efccaa00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efccc06e510 0x7efccc14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.876 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcca000640 1 -- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efccc06e510 msgr2=0x7efccc14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efccb400640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 0x7efccc152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.873+0000 7efcca000640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efccc06e510 0x7efccc14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcca000640 1 -- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 msgr2=0x7efccc152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcca000640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 0x7efccc152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcca000640 1 -- 172.21.15.106:0/1732760919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc156da0 con 0x7efccc0b5910 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efccb400640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 0x7efccc152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcca000640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c7f0 tx=0x7efcb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8015070 con 0x7efccc0b5910 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8004240 con 0x7efccc0b5910 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8007620 con 0x7efccc0b5910 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc153900 con 0x7efccc0b5910 2024-04-26T21:45:43.877 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.874+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc153340 con 0x7efccc0b5910 2024-04-26T21:45:43.878 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.875+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc069e70 con 0x7efccc0b5910 2024-04-26T21:45:43.878 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.876+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7efcb80079d0 con 0x7efccc0b5910 2024-04-26T21:45:43.878 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.877+0000 7efcc3400640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 0x7efcac06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.878 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.877+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7efcb808ad40 con 0x7efccc0b5910 2024-04-26T21:45:43.878 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.877+0000 7efccaa00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 0x7efcac06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.879 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.878+0000 7efccaa00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 0x7efcac06ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcbc034000 tx=0x7efcbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.884 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:43.883+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efcb8059fd0 con 0x7efccc0b5910 2024-04-26T21:45:44.071 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.070+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efccc0af7c0 con 0x7efccc0b5910 2024-04-26T21:45:44.072 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.071+0000 7efcc3400640 1 -- 172.21.15.106:0/1732760919 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7efcb80598d0 con 0x7efccc0b5910 2024-04-26T21:45:44.072 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-26T21:45:44.073 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":12,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:41.117305+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:44.075 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 msgr2=0x7efcac06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.075 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 0x7efcac06ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcbc034000 tx=0x7efcbc02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:44.075 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 msgr2=0x7efccc1505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.075 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c7f0 tx=0x7efcb800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 shutdown_connections 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efccc15f260 0x7efccc152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efccc0b5910 0x7efccc1505d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efcac06c890 0x7efcac06ed50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 --2- 172.21.15.106:0/1732760919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efccc06e510 0x7efccc14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 >> 172.21.15.106:0/1732760919 conn(0x7efccc0d0c40 msgr2=0x7efccc0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 shutdown_connections 2024-04-26T21:45:44.076 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:44.074+0000 7efcd1e00640 1 -- 172.21.15.106:0/1732760919 wait complete. 2024-04-26T21:45:44.089 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:45:44.811 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:44.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:50.896 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:51.084 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.082+0000 7fe411e00640 1 Processor -- start 2024-04-26T21:45:51.084 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.082+0000 7fe411e00640 1 -- start start 2024-04-26T21:45:51.085 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe40c0afb30 0x7fe40c167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.085 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 0x7fe40c0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.085 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0738a0 0x7fe40c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.085 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe40c168040 con 0x7fe40c0afb30 2024-04-26T21:45:51.085 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe40c071ce0 con 0x7fe40c0aff10 2024-04-26T21:45:51.086 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe411e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c0d0140 con 0x7fe40c0738a0 2024-04-26T21:45:51.086 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe410a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe40c0afb30 0x7fe40c167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.086 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe410a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe40c0afb30 0x7fe40c167b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40828/0 (socket says 172.21.15.106:40828) 2024-04-26T21:45:51.086 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.084+0000 7fe410a00640 1 -- 172.21.15.106:0/2990041452 learned_addr learned my addr 172.21.15.106:0/2990041452 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe403e00640 1 --2- 172.21.15.106:0/2990041452 >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 0x7fe40c0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe411400640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0738a0 0x7fe40c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe410a00640 1 -- 172.21.15.106:0/2990041452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0738a0 msgr2=0x7fe40c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe410a00640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0738a0 0x7fe40c0cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe410a00640 1 -- 172.21.15.106:0/2990041452 >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 msgr2=0x7fe40c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe410a00640 1 --2- 172.21.15.106:0/2990041452 >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 0x7fe40c0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe410a00640 1 -- 172.21.15.106:0/2990041452 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c073d80 con 0x7fe40c0afb30 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.085+0000 7fe403e00640 1 --2- 172.21.15.106:0/2990041452 >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 0x7fe40c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:51.087 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.086+0000 7fe410a00640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c167b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0ce600 tx=0x7fe3fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabb7ee38c2478b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.088 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.086+0000 7fe40be00640 1 -- 172.21.15.106:0/2990041452 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc037070 con 0x7fe40c0afb30 2024-04-26T21:45:51.088 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.086+0000 7fe40be00640 1 -- 172.21.15.106:0/2990041452 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3fc032440 con 0x7fe40c0afb30 2024-04-26T21:45:51.088 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.086+0000 7fe40be00640 1 -- 172.21.15.106:0/2990041452 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc032810 con 0x7fe40c0afb30 2024-04-26T21:45:51.088 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 -- 172.21.15.106:0/2990041452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 msgr2=0x7fe40c167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c167b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0ce600 tx=0x7fe3fc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 -- 172.21.15.106:0/2990041452 shutdown_connections 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0738a0 0x7fe40c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 --2- 172.21.15.106:0/2990041452 >> v2:172.21.15.204:3300/0 conn(0x7fe40c0aff10 0x7fe40c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 --2- 172.21.15.106:0/2990041452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c167b00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 -- 172.21.15.106:0/2990041452 >> 172.21.15.106:0/2990041452 conn(0x7fe40c0b25c0 msgr2=0x7fe40c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 -- 172.21.15.106:0/2990041452 shutdown_connections 2024-04-26T21:45:51.089 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.087+0000 7fe411e00640 1 -- 172.21.15.106:0/2990041452 wait complete. 2024-04-26T21:45:51.090 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.089+0000 7fe411e00640 1 Processor -- start 2024-04-26T21:45:51.090 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.089+0000 7fe411e00640 1 -- start start 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe40c0738a0 0x7fe40c1ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0aff10 0x7fe40c163bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0c42e0 con 0x7fe40c0afb30 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c072210 con 0x7fe40c0738a0 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.090+0000 7fe411e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c0c4460 con 0x7fe40c0aff10 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40838/0 (socket says 172.21.15.106:40838) 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 -- 172.21.15.106:0/3013577625 learned_addr learned my addr 172.21.15.106:0/3013577625 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe410a00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe40c0738a0 0x7fe40c1ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 -- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0aff10 msgr2=0x7fe40c163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.092 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0aff10 0x7fe40c163bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.093 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 -- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe40c0738a0 msgr2=0x7fe40c1ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.093 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe40c0738a0 0x7fe40c1ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.093 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 -- 172.21.15.106:0/3013577625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c0c6e40 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.091+0000 7fe403e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c820 tx=0x7fe3f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.092+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4015070 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.092+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4007a30 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.092+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4007e00 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.092+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c1ee680 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.092+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1eec40 con 0x7fe40c0afb30 2024-04-26T21:45:51.094 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.093+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c069cf0 con 0x7fe40c0afb30 2024-04-26T21:45:51.095 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.094+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3f4018070 con 0x7fe40c0afb30 2024-04-26T21:45:51.097 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.096+0000 7fe409600640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.097 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.096+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe3f408b750 con 0x7fe40c0afb30 2024-04-26T21:45:51.097 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.096+0000 7fe410a00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.097 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.097+0000 7fe410a00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 0x7fe3e806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc034000 tx=0x7fe3fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.103 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.102+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3f4059ed0 con 0x7fe40c0afb30 2024-04-26T21:45:51.303 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.302+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe40c0ccb90 con 0x7fe40c0afb30 2024-04-26T21:45:51.305 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.304+0000 7fe409600640 1 -- 172.21.15.106:0/3013577625 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7fe3f40597d0 con 0x7fe40c0afb30 2024-04-26T21:45:51.305 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-26T21:45:51.305 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":16,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:47.506563+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.306+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 msgr2=0x7fe3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.306+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 0x7fe3e806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc034000 tx=0x7fe3fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.306+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 msgr2=0x7fe40c15d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.306+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c820 tx=0x7fe3f400cc20 comp rx=0 tx=0).stop 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 shutdown_connections 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe40c0aff10 0x7fe40c163bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe40c0afb30 0x7fe40c15d0f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.308 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 --2- 172.21.15.106:0/3013577625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe40c0738a0 0x7fe40c1ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.309 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 >> 172.21.15.106:0/3013577625 conn(0x7fe40c0b25c0 msgr2=0x7fe40c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:51.309 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 shutdown_connections 2024-04-26T21:45:51.309 INFO:teuthology.misc.health.smithi106.stderr:2024-04-26T21:45:51.307+0000 7fe411e00640 1 -- 172.21.15.106:0/3013577625 wait complete. 2024-04-26T21:45:51.321 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T21:45:51.322 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T21:45:51.322 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:45:51.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:45:51.330 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:45:51.330 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:45:51.330 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:45:51.337 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:45:51.337 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:45:51.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:45:51.344 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:45:51.344 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:45:51.345 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:45:51.352 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:45:51.352 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T21:45:51.353 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:51.469 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-26T21:45:51.474 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-26T21:45:51.509 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-26T21:45:51.514 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-04-26T21:45:51.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.523+0000 7f086b200640 1 Processor -- start 2024-04-26T21:45:51.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.523+0000 7f086b200640 1 -- start start 2024-04-26T21:45:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f086415f1b0 0x7f08640d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 0x7f0864162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086b200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0864163910 con 0x7f0864163480 2024-04-26T21:45:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f08640d3f50 con 0x7f086415f1b0 2024-04-26T21:45:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f08640d36b0 con 0x7f0864162b00 2024-04-26T21:45:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086a800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0864163480 0x7f08640cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49772/0 (socket says 172.21.15.106:49772) 2024-04-26T21:45:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f086a800640 1 -- 172.21.15.106:0/2981002611 learned_addr learned my addr 172.21.15.106:0/2981002611 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f0869e00640 1 --2- 172.21.15.106:0/2981002611 >> v2:172.21.15.204:3300/0 conn(0x7f086415f1b0 0x7f08640d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.524+0000 7f0869400640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 0x7f0864162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0868a00640 1 -- 172.21.15.106:0/2981002611 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317116540 0 0) 0x7f0864163910 con 0x7f0864163480 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0868a00640 1 -- 172.21.15.106:0/2981002611 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f084c003520 con 0x7f0864163480 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0869e00640 1 -- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 msgr2=0x7f0864162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0869e00640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 0x7f0864162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0869e00640 1 -- 172.21.15.106:0/2981002611 >> v1:172.21.15.106:6789/0 conn(0x7f0864163480 legacy=0x7f08640cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0869e00640 1 -- 172.21.15.106:0/2981002611 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08640d3990 con 0x7f086415f1b0 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.525+0000 7f0869400640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 0x7f0864162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.526+0000 7f0869e00640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f08640d5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08580097a0 tx=0x7f085802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14dd38653c05f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.526+0000 7f0868a00640 1 -- 172.21.15.106:0/2981002611 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858037070 con 0x7f086415f1b0 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.526+0000 7f0868a00640 1 -- 172.21.15.106:0/2981002611 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858032440 con 0x7f086415f1b0 2024-04-26T21:45:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.526+0000 7f0868a00640 1 -- 172.21.15.106:0/2981002611 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858032810 con 0x7f086415f1b0 2024-04-26T21:45:51.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 -- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 msgr2=0x7f08640d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f08640d5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08580097a0 tx=0x7f085802a180 comp rx=0 tx=0).stop 2024-04-26T21:45:51.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 -- 172.21.15.106:0/2981002611 shutdown_connections 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864162b00 0x7f0864162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 --2- 172.21.15.106:0/2981002611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f08640d5990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 -- 172.21.15.106:0/2981002611 >> 172.21.15.106:0/2981002611 conn(0x7f086409e230 msgr2=0x7f08640d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 -- 172.21.15.106:0/2981002611 shutdown_connections 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.527+0000 7f086b200640 1 -- 172.21.15.106:0/2981002611 wait complete. 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.528+0000 7f086b200640 1 Processor -- start 2024-04-26T21:45:51.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 -- start start 2024-04-26T21:45:51.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f086415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864163480 0x7f086415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f08640d33a0 con 0x7f0864162b00 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08640b7d90 con 0x7f086415f1b0 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.529+0000 7f086b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f08640b92c0 con 0x7f0864163480 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40900/0 (socket says 172.21.15.106:40900) 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 -- 172.21.15.106:0/2539682062 learned_addr learned my addr 172.21.15.106:0/2539682062 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869e00640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f086415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f086a800640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864163480 0x7f086415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 -- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864163480 msgr2=0x7f086415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864163480 0x7f086415ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 -- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 msgr2=0x7f086415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f086415a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 -- 172.21.15.106:0/2539682062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08640cb360 con 0x7f0864162b00 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869e00640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f086415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.530+0000 7f0869400640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f085400ec10 tx=0x7f085400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.531+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854017070 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.531+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085400cc30 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.531+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854013470 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.531+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08640ca2c0 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.531+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08640ca880 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.532+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08640693c0 con 0x7f0864162b00 2024-04-26T21:45:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.533+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f085401a070 con 0x7f0864162b00 2024-04-26T21:45:51.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.533+0000 7f085e000640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 0x7f084806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.534+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f085408b760 con 0x7f0864162b00 2024-04-26T21:45:51.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.534+0000 7f0869e00640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 0x7f084806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.534+0000 7f0869e00640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 0x7f084806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0858034000 tx=0x7f085802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.538+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08540597f0 con 0x7f0864162b00 2024-04-26T21:45:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.733+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f086408d090 con 0x7f0864162b00 2024-04-26T21:45:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.734+0000 7f085e000640 1 -- 172.21.15.106:0/2539682062 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f085405daf0 con 0x7f0864162b00 2024-04-26T21:45:51.736 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:51.736 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:47.506563+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:51.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 msgr2=0x7f084806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 0x7f084806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0858034000 tx=0x7f085802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:51.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 msgr2=0x7f08640c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f085400ec10 tx=0x7f085400c520 comp rx=0 tx=0).stop 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 shutdown_connections 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f084806c8f0 0x7f084806edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0864163480 0x7f086415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0864162b00 0x7f08640c9950 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 --2- 172.21.15.106:0/2539682062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f086415f1b0 0x7f086415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 >> 172.21.15.106:0/2539682062 conn(0x7f086409e230 msgr2=0x7f08641615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 shutdown_connections 2024-04-26T21:45:51.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.737+0000 7f086b200640 1 -- 172.21.15.106:0/2539682062 wait complete. 2024-04-26T21:45:51.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:45:51.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:45:51.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.951+0000 7fe273e00640 1 Processor -- start 2024-04-26T21:45:51.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.951+0000 7fe273e00640 1 -- start start 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe273e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe274160d60 0x7fe2740b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe273e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe2741466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe273e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2740b4d20 con 0x7fe274161140 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe273e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe27406e5f0 con 0x7fe274160d60 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe273e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe274146c30 con 0x7fe274146280 2024-04-26T21:45:51.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe26be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe274161140 0x7fe27406dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49778/0 (socket says 172.21.15.106:49778) 2024-04-26T21:45:51.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.953+0000 7fe26be00640 1 -- 172.21.15.106:0/680470228 learned_addr learned my addr 172.21.15.106:0/680470228 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272a00640 1 --2- 172.21.15.106:0/680470228 >> v2:172.21.15.204:3300/0 conn(0x7fe274160d60 0x7fe2740b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158966704 0 0) 0x7fe2740b4d20 con 0x7fe274161140 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe273400640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe2741466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe258003520 con 0x7fe274161140 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 989272470 0 0) 0x7fe258003520 con 0x7fe274161140 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 msgr2=0x7fe2741466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe2741466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 >> v2:172.21.15.204:3300/0 conn(0x7fe274160d60 msgr2=0x7fe2740b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 --2- 172.21.15.106:0/680470228 >> v2:172.21.15.204:3300/0 conn(0x7fe274160d60 0x7fe2740b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2740b5000 con 0x7fe274161140 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fe25c001670 con 0x7fe274161140 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 >> v1:172.21.15.106:6789/0 conn(0x7fe274161140 legacy=0x7fe27406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe258003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe258003ff0 0x7fe258004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe258004d40 0x7fe2580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe27406e5f0 con 0x7fe258004d40 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe274146c30 con 0x7fe258003ff0 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.954+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe25800baf0 con 0x7fe274146280 2024-04-26T21:45:51.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe258004d40 0x7fe2580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe272a00640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe258003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 -- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 msgr2=0x7fe258003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe274146280 0x7fe258003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 -- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe258003ff0 msgr2=0x7fe258004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe258003ff0 0x7fe258004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.955+0000 7fe273400640 1 -- 172.21.15.106:0/680470228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe258003520 con 0x7fe258004d40 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.956+0000 7fe273400640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe258004d40 0x7fe2580051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009fc0 tx=0x7fe258005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f28ad71f2842ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.956+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c009c10 con 0x7fe258004d40 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.956+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c036070 con 0x7fe258004d40 2024-04-26T21:45:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.956+0000 7fe272000640 1 -- 172.21.15.106:0/680470228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c031880 con 0x7fe258004d40 2024-04-26T21:45:51.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 -- 172.21.15.106:0/680470228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe258004d40 msgr2=0x7fe2580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 --2- 172.21.15.106:0/680470228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe258004d40 0x7fe2580051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009fc0 tx=0x7fe258005830 comp rx=0 tx=0).stop 2024-04-26T21:45:51.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273400640 1 -- 172.21.15.106:0/680470228 reap_dead start 2024-04-26T21:45:51.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 -- 172.21.15.106:0/680470228 shutdown_connections 2024-04-26T21:45:51.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 -- 172.21.15.106:0/680470228 >> 172.21.15.106:0/680470228 conn(0x7fe2740b2900 msgr2=0x7fe2740b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 -- 172.21.15.106:0/680470228 shutdown_connections 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.957+0000 7fe273e00640 1 -- 172.21.15.106:0/680470228 wait complete. 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.959+0000 7fe273e00640 1 Processor -- start 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.959+0000 7fe273e00640 1 -- start start 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 0x7fe2741546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe27413bb20 0x7fe2741407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2740cf6f0 con 0x7fe27413bfa0 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2740cf410 con 0x7fe27413bb20 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe2740b6070 con 0x7fe258004d40 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40916/0 (socket says 172.21.15.106:40916) 2024-04-26T21:45:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 -- 172.21.15.106:0/678635978 learned_addr learned my addr 172.21.15.106:0/678635978 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe272a00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 0x7fe2741546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe26be00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe27413bb20 0x7fe2741407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 -- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 msgr2=0x7fe2741546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 0x7fe2741546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 -- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe27413bb20 msgr2=0x7fe2741407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe27413bb20 0x7fe2741407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:51.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe273400640 1 -- 172.21.15.106:0/678635978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe274072a30 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.960+0000 7fe272a00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 0x7fe2741546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe273400640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00e230 tx=0x7fe26c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c009c10 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c031df0 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c003bd0 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe274072cd0 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.961+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe274073290 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.962+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2740693c0 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.963+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe26c005d10 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.964+0000 7fe26aa00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 0x7fe25006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.964+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe26c0acbe0 con 0x7fe27413bfa0 2024-04-26T21:45:51.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.964+0000 7fe272a00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 0x7fe25006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:51.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.965+0000 7fe272a00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 0x7fe25006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:51.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:51.968+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe26c04a060 con 0x7fe27413bfa0 2024-04-26T21:45:52.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.157+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe274143920 con 0x7fe27413bfa0 2024-04-26T21:45:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.158+0000 7fe26aa00640 1 -- 172.21.15.106:0/678635978 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fe26c07b2c0 con 0x7fe27413bfa0 2024-04-26T21:45:52.159 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.160+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 msgr2=0x7fe25006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.160+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 0x7fe25006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.160+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 msgr2=0x7fe274140cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.160+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00e230 tx=0x7fe26c009750 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 shutdown_connections 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe27413bfa0 0x7fe274140cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe27413bb20 0x7fe2741407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe25006c8f0 0x7fe25006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 --2- 172.21.15.106:0/678635978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe258004d40 0x7fe2741546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 >> 172.21.15.106:0/678635978 conn(0x7fe2740b2900 msgr2=0x7fe27415f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:52.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 shutdown_connections 2024-04-26T21:45:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.161+0000 7fe273e00640 1 -- 172.21.15.106:0/678635978 wait complete. 2024-04-26T21:45:52.179 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.369+0000 7f08eec00640 1 Processor -- start 2024-04-26T21:45:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.369+0000 7f08eec00640 1 -- start start 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08eec00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08e815cdf0 0x7f08e80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08eec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 0x7f08e806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08eec00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e80af9a0 con 0x7f08e80b4f50 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08eec00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f08e80ae2c0 con 0x7f08e815cdf0 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08eec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f08e80b05b0 con 0x7f08e80cd650 2024-04-26T21:45:52.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08e7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08e815cdf0 0x7f08e80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08e7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08e815cdf0 0x7f08e80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40928/0 (socket says 172.21.15.106:40928) 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.371+0000 7f08e7e00640 1 -- 172.21.15.106:0/2350172243 learned_addr learned my addr 172.21.15.106:0/2350172243 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7400640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 0x7f08e806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7e00640 1 -- 172.21.15.106:0/2350172243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 msgr2=0x7f08e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7e00640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 0x7f08e806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7e00640 1 -- 172.21.15.106:0/2350172243 >> v1:172.21.15.106:6789/0 conn(0x7f08e80b4f50 legacy=0x7f08e8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7e00640 1 -- 172.21.15.106:0/2350172243 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e80afc80 con 0x7f08e815cdf0 2024-04-26T21:45:52.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.372+0000 7f08e7400640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 0x7f08e806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:52.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.373+0000 7f08e7e00640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80add80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009870 tx=0x7f08d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f979f30585e48a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.373+0000 7f08e6a00640 1 -- 172.21.15.106:0/2350172243 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8037070 con 0x7f08e815cdf0 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.373+0000 7f08e6a00640 1 -- 172.21.15.106:0/2350172243 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d8032440 con 0x7f08e815cdf0 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.373+0000 7f08e6a00640 1 -- 172.21.15.106:0/2350172243 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8032810 con 0x7f08e815cdf0 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 -- 172.21.15.106:0/2350172243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 msgr2=0x7f08e80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80add80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009870 tx=0x7f08d802a250 comp rx=0 tx=0).stop 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 -- 172.21.15.106:0/2350172243 shutdown_connections 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80cd650 0x7f08e806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 --2- 172.21.15.106:0/2350172243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80add80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 -- 172.21.15.106:0/2350172243 >> 172.21.15.106:0/2350172243 conn(0x7f08e80b2d50 msgr2=0x7f08e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:52.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 -- 172.21.15.106:0/2350172243 shutdown_connections 2024-04-26T21:45:52.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.374+0000 7f08eec00640 1 -- 172.21.15.106:0/2350172243 wait complete. 2024-04-26T21:45:52.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.375+0000 7f08eec00640 1 Processor -- start 2024-04-26T21:45:52.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 -- start start 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80b4f50 0x7f08e8158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e80cd650 0x7f08e80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f08e815d1d0 con 0x7f08e815cdf0 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08e80b0240 con 0x7f08e80cd650 2024-04-26T21:45:52.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.376+0000 7f08eec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f08e815d350 con 0x7f08e80b4f50 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40940/0 (socket says 172.21.15.106:40940) 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 -- 172.21.15.106:0/804508109 learned_addr learned my addr 172.21.15.106:0/804508109 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08e7e00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80b4f50 0x7f08e8158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 -- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80b4f50 msgr2=0x7f08e8158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80b4f50 0x7f08e8158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 -- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e80cd650 msgr2=0x7f08e80ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e80cd650 0x7f08e80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 -- 172.21.15.106:0/804508109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e81f53e0 con 0x7f08e815cdf0 2024-04-26T21:45:52.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.377+0000 7f08ecc00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c6e0 tx=0x7f08e000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.378+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003b70 con 0x7f08e815cdf0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.378+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e0003d30 con 0x7f08e815cdf0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.378+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e81f56c0 con 0x7f08e815cdf0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.378+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0005310 con 0x7f08e815cdf0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.378+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e81f5c80 con 0x7f08e815cdf0 2024-04-26T21:45:52.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.379+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e8069540 con 0x7f08e815cdf0 2024-04-26T21:45:52.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.380+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08e00054b0 con 0x7f08e815cdf0 2024-04-26T21:45:52.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.381+0000 7f08dfe00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.381+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f08e008b780 con 0x7f08e815cdf0 2024-04-26T21:45:52.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.381+0000 7f08e7e00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.382+0000 7f08e7e00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 0x7f08c806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08d8034000 tx=0x7f08d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.386+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08e0059e60 con 0x7f08e815cdf0 2024-04-26T21:45:52.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.630+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08e81f5e50 con 0x7f08e815cdf0 2024-04-26T21:45:52.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.631+0000 7f08dfe00640 1 -- 172.21.15.106:0/804508109 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3874 (secure 0 0 0) 0x7f08e0059760 con 0x7f08e815cdf0 2024-04-26T21:45:52.633 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:52.633 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:45:52.633 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2024-04-26T21:45:52.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 msgr2=0x7f08c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 0x7f08c806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08d8034000 tx=0x7f08d802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:52.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 msgr2=0x7f08e80cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c6e0 tx=0x7f08e000cba0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 shutdown_connections 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08e815cdf0 0x7f08e80cf2e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e80cd650 0x7f08e80ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 --2- 172.21.15.106:0/804508109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08e80b4f50 0x7f08e8158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 >> 172.21.15.106:0/804508109 conn(0x7f08e80b2d50 msgr2=0x7f08e80b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 shutdown_connections 2024-04-26T21:45:52.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.634+0000 7f08eec00640 1 -- 172.21.15.106:0/804508109 wait complete. 2024-04-26T21:45:52.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:52.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.844+0000 7f85ad000640 1 Processor -- start 2024-04-26T21:45:52.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.844+0000 7f85ad000640 1 -- start start 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.844+0000 7f85ad000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a8146680 0x7f85a80b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.844+0000 7f85ad000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f85a80b4b30 0x7f85a80d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.844+0000 7f85ad000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a8146a60 con 0x7f85a80d0e70 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85ad000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f85a806e970 con 0x7f85a80b4b30 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85ad000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f85a80d17f0 con 0x7f85a8146680 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f85a80b4b30 0x7f85a80d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f85a80b4b30 0x7f85a80d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40944/0 (socket says 172.21.15.106:40944) 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 -- 172.21.15.106:0/546525318 learned_addr learned my addr 172.21.15.106:0/546525318 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:52.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7400640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a8146680 0x7f85a80b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 -- 172.21.15.106:0/546525318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a8146680 msgr2=0x7f85a80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a8146680 0x7f85a80b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 -- 172.21.15.106:0/546525318 >> v1:172.21.15.204:6789/0 conn(0x7f85a80d0e70 legacy=0x7f85a8160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.845+0000 7f85a7e00640 1 -- 172.21.15.106:0/546525318 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a806dd60 con 0x7f85a80b4b30 2024-04-26T21:45:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.847+0000 7f85a7e00640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a80d12b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009e50 tx=0x7f85a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc83b40d833856ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.847+0000 7f85a6000640 1 -- 172.21.15.106:0/546525318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0036070 con 0x7f85a80b4b30 2024-04-26T21:45:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.847+0000 7f85a6000640 1 -- 172.21.15.106:0/546525318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a0031600 con 0x7f85a80b4b30 2024-04-26T21:45:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.847+0000 7f85a6000640 1 -- 172.21.15.106:0/546525318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00319d0 con 0x7f85a80b4b30 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 -- 172.21.15.106:0/546525318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 msgr2=0x7f85a80d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a80d12b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009e50 tx=0x7f85a002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 -- 172.21.15.106:0/546525318 shutdown_connections 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a80d12b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 --2- 172.21.15.106:0/546525318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a8146680 0x7f85a80b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 -- 172.21.15.106:0/546525318 >> 172.21.15.106:0/546525318 conn(0x7f85a806c740 msgr2=0x7f85a806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 -- 172.21.15.106:0/546525318 shutdown_connections 2024-04-26T21:45:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.848+0000 7f85ad000640 1 -- 172.21.15.106:0/546525318 wait complete. 2024-04-26T21:45:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.850+0000 7f85ad000640 1 Processor -- start 2024-04-26T21:45:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.850+0000 7f85ad000640 1 -- start start 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a80d0e70 0x7f85a813f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 0x7f85a81f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85a806e600 con 0x7f85a80b4b30 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85a806e320 con 0x7f85a8146680 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85ad000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f85a80cd6a0 con 0x7f85a80d0e70 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.851+0000 7f85a7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40958/0 (socket says 172.21.15.106:40958) 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 -- 172.21.15.106:0/1039215343 learned_addr learned my addr 172.21.15.106:0/1039215343 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:52.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7e00640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 0x7f85a81f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 -- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a80d0e70 msgr2=0x7f85a813f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a6a00640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a80d0e70 0x7f85a813f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a80d0e70 0x7f85a813f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 -- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 msgr2=0x7f85a81f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 0x7f85a81f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 -- 172.21.15.106:0/1039215343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a813e830 con 0x7f85a80b4b30 2024-04-26T21:45:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7e00640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 0x7f85a81f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.852+0000 7f85a7400640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f859800d5f0 tx=0x7f859800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.853+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980125f0 con 0x7f85a80b4b30 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.853+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8598019070 con 0x7f85a80b4b30 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.853+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980129d0 con 0x7f85a80b4b30 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.853+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a8141db0 con 0x7f85a80b4b30 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.853+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81417f0 con 0x7f85a80b4b30 2024-04-26T21:45:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.854+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a80693c0 con 0x7f85a80b4b30 2024-04-26T21:45:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.855+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8598012b70 con 0x7f85a80b4b30 2024-04-26T21:45:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.856+0000 7f859f400640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 0x7f858806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.856+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f859808bcf0 con 0x7f85a80b4b30 2024-04-26T21:45:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.856+0000 7f85a6a00640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 0x7f858806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.856+0000 7f85a6a00640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 0x7f858806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85a8137e40 tx=0x7f8594009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:52.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:52.860+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f859805a3d0 con 0x7f85a80b4b30 2024-04-26T21:45:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.044+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85a81605d0 con 0x7f85a80b4b30 2024-04-26T21:45:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.046+0000 7f859f400640 1 -- 172.21.15.106:0/1039215343 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f8598059cd0 con 0x7f85a80b4b30 2024-04-26T21:45:53.047 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:53.047 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:47.506563+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.048+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 msgr2=0x7f858806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.048+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 0x7f858806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85a8137e40 tx=0x7f8594009040 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.048+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 msgr2=0x7f85a815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.048+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f859800d5f0 tx=0x7f859800dab0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 shutdown_connections 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a8146680 0x7f85a81f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f858806c8f0 0x7f858806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85a80d0e70 0x7f85a813f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 --2- 172.21.15.106:0/1039215343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85a80b4b30 0x7f85a815c220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 >> 172.21.15.106:0/1039215343 conn(0x7f85a806c740 msgr2=0x7f85a80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 shutdown_connections 2024-04-26T21:45:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.049+0000 7f85ad000640 1 -- 172.21.15.106:0/1039215343 wait complete. 2024-04-26T21:45:53.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:45:53.069 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:45:53.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.270+0000 7fabffa00640 1 Processor -- start 2024-04-26T21:45:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.270+0000 7fabffa00640 1 -- start start 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.271+0000 7fabffa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fabf80d19c0 0x7fabf80d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabffa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabffa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fabf80d1da0 0x7fabf8162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabffa00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fabf80d40d0 con 0x7fabf80d19c0 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabffa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fabf80d3830 con 0x7fabf80d1da0 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabffa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fabf80d3b10 con 0x7fabf80b6170 2024-04-26T21:45:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fabf80d19c0 0x7fabf80d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fabf80d19c0 0x7fabf80d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40960/0 (socket says 172.21.15.106:40960) 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 -- 172.21.15.106:0/2245541580 learned_addr learned my addr 172.21.15.106:0/2245541580 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 -- 172.21.15.106:0/2245541580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 msgr2=0x7fabf80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 --2- 172.21.15.106:0/2245541580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 -- 172.21.15.106:0/2245541580 >> v2:172.21.15.204:3300/0 conn(0x7fabf80d1da0 msgr2=0x7fabf8162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 --2- 172.21.15.106:0/2245541580 >> v2:172.21.15.204:3300/0 conn(0x7fabf80d1da0 0x7fabf8162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.272+0000 7fabfd000640 1 -- 172.21.15.106:0/2245541580 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf80d3df0 con 0x7fabf80d19c0 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.273+0000 7fabfd000640 1 --2- 172.21.15.106:0/2245541580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf80d2f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabf80cc160 tx=0x7fabe802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71561d2c53d3f0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabf7e00640 1 -- 172.21.15.106:0/2245541580 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8037070 con 0x7fabf80d19c0 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabf7e00640 1 -- 172.21.15.106:0/2245541580 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe8032440 con 0x7fabf80d19c0 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabf7e00640 1 -- 172.21.15.106:0/2245541580 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8032810 con 0x7fabf80d19c0 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 -- 172.21.15.106:0/2245541580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 msgr2=0x7fabf80d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 --2- 172.21.15.106:0/2245541580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf80d2f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabf80cc160 tx=0x7fabe802a140 comp rx=0 tx=0).stop 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 -- 172.21.15.106:0/2245541580 shutdown_connections 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 --2- 172.21.15.106:0/2245541580 >> v2:172.21.15.204:3300/0 conn(0x7fabf80d1da0 0x7fabf8162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 --2- 172.21.15.106:0/2245541580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf80c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 --2- 172.21.15.106:0/2245541580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf80d2f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 -- 172.21.15.106:0/2245541580 >> 172.21.15.106:0/2245541580 conn(0x7fabf809e460 msgr2=0x7fabf80d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 -- 172.21.15.106:0/2245541580 shutdown_connections 2024-04-26T21:45:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.274+0000 7fabffa00640 1 -- 172.21.15.106:0/2245541580 wait complete. 2024-04-26T21:45:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.276+0000 7fabffa00640 1 Processor -- start 2024-04-26T21:45:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.276+0000 7fabffa00640 1 -- start start 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf81f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf80d1da0 0x7fabf814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fabf80cfc30 con 0x7fabf80d19c0 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fabf80c91b0 con 0x7fabf80d1da0 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabffa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fabf8162b80 con 0x7fabf80b6170 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40966/0 (socket says 172.21.15.106:40966) 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.277+0000 7fabefe00640 1 -- 172.21.15.106:0/2504279826 learned_addr learned my addr 172.21.15.106:0/2504279826 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabfda00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf80d1da0 0x7fabf814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabfd000640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf81f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 -- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 msgr2=0x7fabf81f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf81f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 -- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf80d1da0 msgr2=0x7fabf814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf80d1da0 0x7fabf814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 -- 172.21.15.106:0/2504279826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf8158e20 con 0x7fabf80d19c0 2024-04-26T21:45:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabefe00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b4a0 tx=0x7fabe000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0014070 con 0x7fabf80d19c0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0010610 con 0x7fabf80d19c0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0010a00 con 0x7fabf80d19c0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.278+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf8159100 con 0x7fabf80d19c0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.279+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf814ccb0 con 0x7fabf80d19c0 2024-04-26T21:45:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.280+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf8159620 con 0x7fabf80d19c0 2024-04-26T21:45:53.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.280+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe0017070 con 0x7fabf80d19c0 2024-04-26T21:45:53.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.281+0000 7fabf5600640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 0x7fabd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.281+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fabe008b290 con 0x7fabf80d19c0 2024-04-26T21:45:53.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.282+0000 7fabfd000640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 0x7fabd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.282+0000 7fabfd000640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 0x7fabd406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabe8034000 tx=0x7fabe802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.287+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabf8159620 con 0x7fabf80d19c0 2024-04-26T21:45:53.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.471+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabf8069650 con 0x7fabf80d19c0 2024-04-26T21:45:53.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.472+0000 7fabf5600640 1 -- 172.21.15.106:0/2504279826 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fabe0059970 con 0x7fabf80d19c0 2024-04-26T21:45:53.472 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 msgr2=0x7fabd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 0x7fabd406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabe8034000 tx=0x7fabe802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 msgr2=0x7fabf8158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b4a0 tx=0x7fabe000b960 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 shutdown_connections 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf80d1da0 0x7fabf814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabf80d19c0 0x7fabf8158360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fabd406c920 0x7fabd406ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 --2- 172.21.15.106:0/2504279826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabf80b6170 0x7fabf81f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.474+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 >> 172.21.15.106:0/2504279826 conn(0x7fabf809e460 msgr2=0x7fabf80c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.475+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 shutdown_connections 2024-04-26T21:45:53.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.475+0000 7fabffa00640 1 -- 172.21.15.106:0/2504279826 wait complete. 2024-04-26T21:45:53.491 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:45:53.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.681+0000 7f7d94e00640 1 Processor -- start 2024-04-26T21:45:53.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.681+0000 7f7d94e00640 1 -- start start 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.682+0000 7f7d94e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d90073720 0x7f7d9006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.682+0000 7f7d94e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d9015dec0 0x7f7d90073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d94e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d90074040 con 0x7f7d9006ed40 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d94e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7d900701a0 con 0x7f7d9015dec0 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d94e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d9015e320 con 0x7f7d90073720 2024-04-26T21:45:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7d9006ed40 0x7f7d900cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49798/0 (socket says 172.21.15.106:49798) 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8fe00640 1 -- 172.21.15.106:0/3578779016 learned_addr learned my addr 172.21.15.106:0/3578779016 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8ea00640 1 --2- 172.21.15.106:0/3578779016 >> v2:172.21.15.106:3300/0 conn(0x7f7d9015dec0 0x7f7d90073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8f400640 1 --2- 172.21.15.106:0/3578779016 >> v2:172.21.15.204:3300/0 conn(0x7f7d90073720 0x7f7d9006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8e000640 1 -- 172.21.15.106:0/3578779016 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1247201907 0 0) 0x7f7d90074040 con 0x7f7d9006ed40 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8e000640 1 -- 172.21.15.106:0/3578779016 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d74003520 con 0x7f7d9006ed40 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8ea00640 1 -- 172.21.15.106:0/3578779016 >> v2:172.21.15.204:3300/0 conn(0x7f7d90073720 msgr2=0x7f7d9006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8ea00640 1 --2- 172.21.15.106:0/3578779016 >> v2:172.21.15.204:3300/0 conn(0x7f7d90073720 0x7f7d9006fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8ea00640 1 -- 172.21.15.106:0/3578779016 >> v1:172.21.15.106:6789/0 conn(0x7f7d9006ed40 legacy=0x7f7d900cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.683+0000 7f7d8ea00640 1 -- 172.21.15.106:0/3578779016 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9006df80 con 0x7f7d9015dec0 2024-04-26T21:45:53.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.684+0000 7f7d8ea00640 1 --2- 172.21.15.106:0/3578779016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90073b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d90050f80 tx=0x7f7d7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394174890b8e88a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d8e000640 1 -- 172.21.15.106:0/3578779016 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c037070 con 0x7f7d9015dec0 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d8e000640 1 -- 172.21.15.106:0/3578779016 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c032440 con 0x7f7d9015dec0 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d8e000640 1 -- 172.21.15.106:0/3578779016 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c032810 con 0x7f7d9015dec0 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 -- 172.21.15.106:0/3578779016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 msgr2=0x7f7d90073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 --2- 172.21.15.106:0/3578779016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90073b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d90050f80 tx=0x7f7d7c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 -- 172.21.15.106:0/3578779016 shutdown_connections 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 --2- 172.21.15.106:0/3578779016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90073b00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 --2- 172.21.15.106:0/3578779016 >> v2:172.21.15.204:3300/0 conn(0x7f7d90073720 0x7f7d9006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 -- 172.21.15.106:0/3578779016 >> 172.21.15.106:0/3578779016 conn(0x7f7d900d0ae0 msgr2=0x7f7d900d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 -- 172.21.15.106:0/3578779016 shutdown_connections 2024-04-26T21:45:53.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.685+0000 7f7d94e00640 1 -- 172.21.15.106:0/3578779016 wait complete. 2024-04-26T21:45:53.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.687+0000 7f7d94e00640 1 Processor -- start 2024-04-26T21:45:53.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.687+0000 7f7d94e00640 1 -- start start 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9006ed40 0x7f7d901f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d90073720 0x7f7d900b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006f1f0 con 0x7f7d9015dec0 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006e540 con 0x7f7d9006ed40 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d94e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d90146910 con 0x7f7d90073720 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.688+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9006ed40 0x7f7d901f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40978/0 (socket says 172.21.15.106:40978) 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 -- 172.21.15.106:0/1918670888 learned_addr learned my addr 172.21.15.106:0/1918670888 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8ea00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d90073720 0x7f7d900b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 -- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d90073720 msgr2=0x7f7d900b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d90073720 0x7f7d900b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 -- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9006ed40 msgr2=0x7f7d901f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9006ed40 0x7f7d901f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 -- 172.21.15.106:0/1918670888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900b6380 con 0x7f7d9015dec0 2024-04-26T21:45:53.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.689+0000 7f7d8fe00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800caa0 tx=0x7f7d8800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.690+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88017070 con 0x7f7d9015dec0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.690+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8800aba0 con 0x7f7d9015dec0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.690+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88012400 con 0x7f7d9015dec0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.690+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d90141dc0 con 0x7f7d9015dec0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.690+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d90142380 con 0x7f7d9015dec0 2024-04-26T21:45:53.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.691+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d900b6650 con 0x7f7d9015dec0 2024-04-26T21:45:53.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.692+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d8801a070 con 0x7f7d9015dec0 2024-04-26T21:45:53.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.693+0000 7f7d87400640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 0x7f7d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:53.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.693+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7d8808b5f0 con 0x7f7d9015dec0 2024-04-26T21:45:53.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.693+0000 7f7d8f400640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 0x7f7d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:53.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.694+0000 7f7d8f400640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 0x7f7d7006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006fd0 tx=0x7f7d80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:53.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.697+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d88059680 con 0x7f7d9015dec0 2024-04-26T21:45:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.886+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7d90160540 con 0x7f7d9015dec0 2024-04-26T21:45:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.887+0000 7f7d87400640 1 -- 172.21.15.106:0/1918670888 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f7d8805d980 con 0x7f7d9015dec0 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:45:53.887 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.889+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 msgr2=0x7f7d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.889+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 0x7f7d7006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006fd0 tx=0x7f7d80008040 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.889+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 msgr2=0x7f7d90142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.889+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800caa0 tx=0x7f7d8800cf60 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 shutdown_connections 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d9015dec0 0x7f7d90142b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d7006c8f0 0x7f7d7006edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d90073720 0x7f7d900b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 --2- 172.21.15.106:0/1918670888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9006ed40 0x7f7d901f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 >> 172.21.15.106:0/1918670888 conn(0x7f7d900d0ae0 msgr2=0x7f7d900b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 shutdown_connections 2024-04-26T21:45:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:53.890+0000 7f7d94e00640 1 -- 172.21.15.106:0/1918670888 wait complete. 2024-04-26T21:45:53.907 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:45:53.907 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:45:54.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.111+0000 7fe386600640 1 Processor -- start 2024-04-26T21:45:54.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.111+0000 7fe386600640 1 -- start start 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.112+0000 7fe386600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe38006f9d0 0x7fe38006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.112+0000 7fe386600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3800b5210 0x7fe3800b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.112+0000 7fe386600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3800b5f40 con 0x7fe38015bf50 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.112+0000 7fe386600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe380070370 con 0x7fe38006f9d0 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.112+0000 7fe386600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe38015c330 con 0x7fe3800b5210 2024-04-26T21:45:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe37f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe38015bf50 0x7fe380072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49804/0 (socket says 172.21.15.106:49804) 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe37f400640 1 -- 172.21.15.106:0/3508009387 learned_addr learned my addr 172.21.15.106:0/3508009387 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe376a00640 1 --2- 172.21.15.106:0/3508009387 >> v2:172.21.15.106:3300/0 conn(0x7fe38006f9d0 0x7fe38006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe37ea00640 1 -- 172.21.15.106:0/3508009387 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174279773 0 0) 0x7fe3800b5f40 con 0x7fe38015bf50 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe37fe00640 1 --2- 172.21.15.106:0/3508009387 >> v2:172.21.15.204:3300/0 conn(0x7fe3800b5210 0x7fe3800b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe37ea00640 1 -- 172.21.15.106:0/3508009387 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe364003520 con 0x7fe38015bf50 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe376a00640 1 -- 172.21.15.106:0/3508009387 >> v2:172.21.15.204:3300/0 conn(0x7fe3800b5210 msgr2=0x7fe3800b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe376a00640 1 --2- 172.21.15.106:0/3508009387 >> v2:172.21.15.204:3300/0 conn(0x7fe3800b5210 0x7fe3800b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe376a00640 1 -- 172.21.15.106:0/3508009387 >> v1:172.21.15.106:6789/0 conn(0x7fe38015bf50 legacy=0x7fe380072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.113+0000 7fe376a00640 1 -- 172.21.15.106:0/3508009387 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3800aeaa0 con 0x7fe38006f9d0 2024-04-26T21:45:54.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.114+0000 7fe376a00640 1 --2- 172.21.15.106:0/3508009387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38006f9d0 0x7fe38006fe30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3800b28c0 tx=0x7fe36802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbbfceaeac515bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe37ea00640 1 -- 172.21.15.106:0/3508009387 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368037070 con 0x7fe38006f9d0 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe37ea00640 1 -- 172.21.15.106:0/3508009387 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe368032440 con 0x7fe38006f9d0 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe37ea00640 1 -- 172.21.15.106:0/3508009387 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368032810 con 0x7fe38006f9d0 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 -- 172.21.15.106:0/3508009387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38006f9d0 msgr2=0x7fe38006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 --2- 172.21.15.106:0/3508009387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38006f9d0 0x7fe38006fe30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3800b28c0 tx=0x7fe36802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 -- 172.21.15.106:0/3508009387 shutdown_connections 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 --2- 172.21.15.106:0/3508009387 >> v2:172.21.15.204:3300/0 conn(0x7fe3800b5210 0x7fe3800b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 --2- 172.21.15.106:0/3508009387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38006f9d0 0x7fe38006fe30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 -- 172.21.15.106:0/3508009387 >> 172.21.15.106:0/3508009387 conn(0x7fe3800d0c40 msgr2=0x7fe3800d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 -- 172.21.15.106:0/3508009387 shutdown_connections 2024-04-26T21:45:54.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.115+0000 7fe386600640 1 -- 172.21.15.106:0/3508009387 wait complete. 2024-04-26T21:45:54.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.117+0000 7fe386600640 1 Processor -- start 2024-04-26T21:45:54.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.117+0000 7fe386600640 1 -- start start 2024-04-26T21:45:54.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe38006f9d0 0x7fe380144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 0x7fe38015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3800af340 con 0x7fe38015bf50 2024-04-26T21:45:54.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3800af060 con 0x7fe3800b5210 2024-04-26T21:45:54.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.118+0000 7fe386600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3800b5be0 con 0x7fe38006f9d0 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe376a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 0x7fe38015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe38006f9d0 0x7fe380144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe376a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 0x7fe38015e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:48178/0 (socket says 172.21.15.106:48178) 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40986/0 (socket says 172.21.15.106:40986) 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe376a00640 1 -- 172.21.15.106:0/426439912 learned_addr learned my addr 172.21.15.106:0/426439912 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 -- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe38006f9d0 msgr2=0x7fe380144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe38006f9d0 0x7fe380144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 -- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 msgr2=0x7fe38015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 0x7fe38015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 -- 172.21.15.106:0/426439912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38015d710 con 0x7fe38015bf50 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.119+0000 7fe37fe00640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe378007d90 tx=0x7fe37800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.120+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37800edd0 con 0x7fe38015bf50 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.120+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe378003990 con 0x7fe38015bf50 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.120+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378003d80 con 0x7fe38015bf50 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.120+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38015d9d0 con 0x7fe38015bf50 2024-04-26T21:45:54.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.120+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38015df90 con 0x7fe38015bf50 2024-04-26T21:45:54.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.121+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe380069650 con 0x7fe38015bf50 2024-04-26T21:45:54.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.122+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe37801b020 con 0x7fe38015bf50 2024-04-26T21:45:54.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.123+0000 7fe377e00640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.124+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe37808b860 con 0x7fe38015bf50 2024-04-26T21:45:54.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.124+0000 7fe37f400640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.124+0000 7fe37f400640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 0x7fe35c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe370005880 tx=0x7fe370009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.128+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe378059990 con 0x7fe38015bf50 2024-04-26T21:45:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.328+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe38006e340 con 0x7fe38015bf50 2024-04-26T21:45:54.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.630+0000 7fe377e00640 1 -- 172.21.15.106:0/426439912 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fe37805ee20 con 0x7fe38015bf50 2024-04-26T21:45:54.630 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-26T21:45:54.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.633+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 msgr2=0x7fe35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.633+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 0x7fe35c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe370005880 tx=0x7fe370009040 comp rx=0 tx=0).stop 2024-04-26T21:45:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 msgr2=0x7fe38009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe378007d90 tx=0x7fe37800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 shutdown_connections 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe38015bf50 0x7fe38009ba50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3800b5210 0x7fe38015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe35c06c8f0 0x7fe35c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 --2- 172.21.15.106:0/426439912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe38006f9d0 0x7fe380144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 >> 172.21.15.106:0/426439912 conn(0x7fe3800d0c40 msgr2=0x7fe3800721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 shutdown_connections 2024-04-26T21:45:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.634+0000 7fe386600640 1 -- 172.21.15.106:0/426439912 wait complete. 2024-04-26T21:45:54.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:45:54.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.857+0000 7f3823200640 1 Processor -- start 2024-04-26T21:45:54.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.857+0000 7f3823200640 1 -- start start 2024-04-26T21:45:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.858+0000 7f3823200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f381c0b6330 0x7f381c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.858+0000 7f3823200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c073070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.858+0000 7f3823200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381c0b4d90 con 0x7f381c15f280 2024-04-26T21:45:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.858+0000 7f3823200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f381c0b0b60 con 0x7f381c0b6330 2024-04-26T21:45:54.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.858+0000 7f3823200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f381c15f660 con 0x7f381c072be0 2024-04-26T21:45:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3813400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f381c0b6330 0x7f381c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f381be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f381c15f280 0x7f381c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49816/0 (socket says 172.21.15.106:49816) 2024-04-26T21:45:54.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f381be00640 1 -- 172.21.15.106:0/3202809538 learned_addr learned my addr 172.21.15.106:0/3202809538 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3821200640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3813400640 1 -- 172.21.15.106:0/3202809538 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 msgr2=0x7f381c073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3813400640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3813400640 1 -- 172.21.15.106:0/3202809538 >> v1:172.21.15.106:6789/0 conn(0x7f381c15f280 legacy=0x7f381c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f3813400640 1 -- 172.21.15.106:0/3202809538 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c0b5070 con 0x7f381c0b6330 2024-04-26T21:45:54.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.859+0000 7f381b400640 1 -- 172.21.15.106:0/3202809538 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314577273 0 0) 0x7f381c0b4d90 con 0x7f381c15f280 2024-04-26T21:45:54.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.860+0000 7f3813400640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c0b4690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38040097a0 tx=0x7f380402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5a3d50ff77e3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f381b400640 1 -- 172.21.15.106:0/3202809538 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804037070 con 0x7f381c0b6330 2024-04-26T21:45:54.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f381b400640 1 -- 172.21.15.106:0/3202809538 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380402ae50 con 0x7f381c0b6330 2024-04-26T21:45:54.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f381b400640 1 -- 172.21.15.106:0/3202809538 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804032660 con 0x7f381c0b6330 2024-04-26T21:45:54.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 -- 172.21.15.106:0/3202809538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 msgr2=0x7f381c0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c0b4690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38040097a0 tx=0x7f380402a100 comp rx=0 tx=0).stop 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 -- 172.21.15.106:0/3202809538 shutdown_connections 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 --2- 172.21.15.106:0/3202809538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c0b4690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 -- 172.21.15.106:0/3202809538 >> 172.21.15.106:0/3202809538 conn(0x7f381c06c590 msgr2=0x7f381c06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 -- 172.21.15.106:0/3202809538 shutdown_connections 2024-04-26T21:45:54.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.861+0000 7f3823200640 1 -- 172.21.15.106:0/3202809538 wait complete. 2024-04-26T21:45:54.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.863+0000 7f3823200640 1 Processor -- start 2024-04-26T21:45:54.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.863+0000 7f3823200640 1 -- start start 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 0x7f381c1f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0ae9f0 con 0x7f381c0b6330 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0b5630 con 0x7f381c15f280 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.864+0000 7f3823200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f381c0aef10 con 0x7f381c072be0 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41004/0 (socket says 172.21.15.106:41004) 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 -- 172.21.15.106:0/2816083372 learned_addr learned my addr 172.21.15.106:0/2816083372 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f381be00640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3821200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 0x7f381c1f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 -- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 msgr2=0x7f381c146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 -- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 msgr2=0x7f381c1f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 0x7f381c1f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 -- 172.21.15.106:0/2816083372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c156a90 con 0x7f381c0b6330 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3821200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 0x7f381c1f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.865+0000 7f3813400640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3804035040 tx=0x7f38040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.866+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804041070 con 0x7f381c0b6330 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.866+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380403c670 con 0x7f381c0b6330 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.866+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804037040 con 0x7f381c0b6330 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.866+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c156d70 con 0x7f381c0b6330 2024-04-26T21:45:54.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.866+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c157330 con 0x7f381c0b6330 2024-04-26T21:45:54.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.867+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c069650 con 0x7f381c0b6330 2024-04-26T21:45:54.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.868+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f380403a070 con 0x7f381c0b6330 2024-04-26T21:45:54.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.869+0000 7f3818c00640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 0x7f37f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.869+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f38040abda0 con 0x7f381c0b6330 2024-04-26T21:45:54.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.869+0000 7f381be00640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 0x7f37f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.870+0000 7f381be00640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 0x7f37f806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f380c007ed0 tx=0x7f380c007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:54.874+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f380407a310 con 0x7f381c0b6330 2024-04-26T21:45:55.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.075+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f381c15b4b0 con 0x7f381c0b6330 2024-04-26T21:45:55.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.636+0000 7f3818c00640 1 -- 172.21.15.106:0/2816083372 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f3804079c10 con 0x7f381c0b6330 2024-04-26T21:45:55.636 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.639+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 msgr2=0x7f37f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.639+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 0x7f37f806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f380c007ed0 tx=0x7f380c007e60 comp rx=0 tx=0).stop 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.639+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 msgr2=0x7f381c162010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.639+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3804035040 tx=0x7f38040046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 shutdown_connections 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f381c15f280 0x7f381c1f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f381c0b6330 0x7f381c162010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f37f806c8f0 0x7f37f806edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 --2- 172.21.15.106:0/2816083372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f381c072be0 0x7f381c146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 >> 172.21.15.106:0/2816083372 conn(0x7f381c06c590 msgr2=0x7f381c15cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 shutdown_connections 2024-04-26T21:45:55.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.640+0000 7f3823200640 1 -- 172.21.15.106:0/2816083372 wait complete. 2024-04-26T21:45:55.675 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:45:55.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.850+0000 7f737cc00640 1 Processor -- start 2024-04-26T21:45:55.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.850+0000 7f737cc00640 1 -- start start 2024-04-26T21:45:55.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.851+0000 7f737cc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7378160d60 0x7f73780b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.851+0000 7f737cc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7378161140 0x7f737806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.851+0000 7f737cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f73781466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f737cc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7378146c30 con 0x7f7378160d60 2024-04-26T21:45:55.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f737cc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f73780b4d20 con 0x7f7378161140 2024-04-26T21:45:55.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f737cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f73780b5000 con 0x7f7378146280 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7378160d60 0x7f73780b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7378160d60 0x7f73780b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41006/0 (socket says 172.21.15.106:41006) 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377400640 1 -- 172.21.15.106:0/4082145829 learned_addr learned my addr 172.21.15.106:0/4082145829 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377e00640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f73781466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7376a00640 1 --2- 172.21.15.106:0/4082145829 >> v2:172.21.15.204:3300/0 conn(0x7f7378161140 0x7f737806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377400640 1 -- 172.21.15.106:0/4082145829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 msgr2=0x7f73781466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.852+0000 7f7377400640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f73781466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.853+0000 7f7377400640 1 -- 172.21.15.106:0/4082145829 >> v2:172.21.15.204:3300/0 conn(0x7f7378161140 msgr2=0x7f737806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.853+0000 7f7377400640 1 --2- 172.21.15.106:0/4082145829 >> v2:172.21.15.204:3300/0 conn(0x7f7378161140 0x7f737806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.853+0000 7f7377400640 1 -- 172.21.15.106:0/4082145829 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73780b52e0 con 0x7f7378160d60 2024-04-26T21:45:55.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.854+0000 7f7377400640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378160d60 0x7f73780b5b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7368009870 tx=0x7f736802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bba2ce808cd7a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.854+0000 7f7376000640 1 -- 172.21.15.106:0/4082145829 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368037070 con 0x7f7378160d60 2024-04-26T21:45:55.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.854+0000 7f7376000640 1 -- 172.21.15.106:0/4082145829 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368032440 con 0x7f7378160d60 2024-04-26T21:45:55.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.854+0000 7f7376000640 1 -- 172.21.15.106:0/4082145829 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368032810 con 0x7f7378160d60 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 -- 172.21.15.106:0/4082145829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378160d60 msgr2=0x7f73780b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378160d60 0x7f73780b5b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7368009870 tx=0x7f736802a210 comp rx=0 tx=0).stop 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 -- 172.21.15.106:0/4082145829 shutdown_connections 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f73781466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 --2- 172.21.15.106:0/4082145829 >> v2:172.21.15.204:3300/0 conn(0x7f7378161140 0x7f737806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 --2- 172.21.15.106:0/4082145829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378160d60 0x7f73780b5b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 -- 172.21.15.106:0/4082145829 >> 172.21.15.106:0/4082145829 conn(0x7f73780b2900 msgr2=0x7f73780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 -- 172.21.15.106:0/4082145829 shutdown_connections 2024-04-26T21:45:55.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.855+0000 7f737cc00640 1 -- 172.21.15.106:0/4082145829 wait complete. 2024-04-26T21:45:55.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.856+0000 7f737cc00640 1 Processor -- start 2024-04-26T21:45:55.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 -- start start 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f7378155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7378160d60 0x7f737813b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73780b6070 con 0x7f7378161140 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73780cf6f0 con 0x7f7378160d60 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.857+0000 7f737cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f737806e420 con 0x7f7378146280 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41016/0 (socket says 172.21.15.106:41016) 2024-04-26T21:45:55.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 -- 172.21.15.106:0/74251788 learned_addr learned my addr 172.21.15.106:0/74251788 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377400640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f7378155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7376a00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7378160d60 0x7f737813b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 -- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 msgr2=0x7f7378155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f7378155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 -- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7378160d60 msgr2=0x7f737813b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7378160d60 0x7f737813b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377e00640 1 -- 172.21.15.106:0/74251788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7378159970 con 0x7f7378161140 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.858+0000 7f7377400640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f7378155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f7377e00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7370007d40 tx=0x7f737000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370014420 con 0x7f7378161140 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7370019070 con 0x7f7378161140 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7378159c50 con 0x7f7378161140 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370014820 con 0x7f7378161140 2024-04-26T21:45:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.859+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7378156130 con 0x7f7378161140 2024-04-26T21:45:55.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.860+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737815b0f0 con 0x7f7378161140 2024-04-26T21:45:55.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.861+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73700149c0 con 0x7f7378161140 2024-04-26T21:45:55.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.862+0000 7f736f400640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 0x7f735806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.862+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f737008c150 con 0x7f7378161140 2024-04-26T21:45:55.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.862+0000 7f7377400640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 0x7f735806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.863+0000 7f7377400640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 0x7f735806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7368034000 tx=0x7f736802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:55.867+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f737005a670 con 0x7f7378161140 2024-04-26T21:45:56.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.101+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f73780693c0 con 0x7f7378161140 2024-04-26T21:45:56.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.684+0000 7f736f400640 1 -- 172.21.15.106:0/74251788 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7370059f70 con 0x7f7378161140 2024-04-26T21:45:56.685 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:45:56.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.691+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 msgr2=0x7f735806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.691+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 0x7f735806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7368034000 tx=0x7f736802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:56.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.691+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 msgr2=0x7f7378156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.691+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7370007d40 tx=0x7f737000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 shutdown_connections 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7378161140 0x7f7378156d60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7378160d60 0x7f737813b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f735806c920 0x7f735806ede0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7368034000 tx=0x7f736802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 --2- 172.21.15.106:0/74251788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7378146280 0x7f7378155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.692+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 >> 172.21.15.106:0/74251788 conn(0x7f73780b2900 msgr2=0x7f7378073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:56.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.694+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 shutdown_connections 2024-04-26T21:45:56.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.694+0000 7f737cc00640 1 -- 172.21.15.106:0/74251788 wait complete. 2024-04-26T21:45:56.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:56.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.894+0000 7f7c73a00640 1 Processor -- start 2024-04-26T21:45:56.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.894+0000 7f7c73a00640 1 -- start start 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.895+0000 7f7c73a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.895+0000 7f7c73a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c6c0cd650 0x7f7c6c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.895+0000 7f7c73a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c6c0af9a0 con 0x7f7c6c15cdf0 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.895+0000 7f7c73a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7c6c0ae2c0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.895+0000 7f7c73a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c0b05b0 con 0x7f7c6c06ce80 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.896+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.896+0000 7f7c71a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c6c0cd650 0x7f7c6c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.896+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:38556/0 (socket says 172.21.15.106:38556) 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.896+0000 7f7c63e00640 1 -- 172.21.15.106:0/1939426998 learned_addr learned my addr 172.21.15.106:0/1939426998 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c71a00640 1 -- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 msgr2=0x7f7c6c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c71a00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c71a00640 1 -- 172.21.15.106:0/1939426998 >> v1:172.21.15.204:6789/0 conn(0x7f7c6c15cdf0 legacy=0x7f7c6c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c6be00640 1 -- 172.21.15.106:0/1939426998 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253530176 0 0) 0x7f7c6c0af9a0 con 0x7f7c6c15cdf0 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c71a00640 1 -- 172.21.15.106:0/1939426998 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c0afc80 con 0x7f7c6c0cd650 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.897+0000 7f7c63e00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:56.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.898+0000 7f7c71a00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c0b3d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009d90 tx=0x7f7c6402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab988e6aab23207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c6be00640 1 -- 172.21.15.106:0/1939426998 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64007740 con 0x7f7c6c0cd650 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c6be00640 1 -- 172.21.15.106:0/1939426998 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c640078e0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c6be00640 1 -- 172.21.15.106:0/1939426998 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64007cb0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c73a00640 1 -- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c0cd650 msgr2=0x7f7c6c0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c73a00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c0b3d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009d90 tx=0x7f7c6402a660 comp rx=0 tx=0).stop 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c73a00640 1 -- 172.21.15.106:0/1939426998 shutdown_connections 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.899+0000 7f7c73a00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c0b3d10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.900+0000 7f7c73a00640 1 --2- 172.21.15.106:0/1939426998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.900+0000 7f7c73a00640 1 -- 172.21.15.106:0/1939426998 >> 172.21.15.106:0/1939426998 conn(0x7f7c6c0b2d50 msgr2=0x7f7c6c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.900+0000 7f7c73a00640 1 -- 172.21.15.106:0/1939426998 shutdown_connections 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.900+0000 7f7c73a00640 1 -- 172.21.15.106:0/1939426998 wait complete. 2024-04-26T21:45:56.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.901+0000 7f7c73a00640 1 Processor -- start 2024-04-26T21:45:56.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.902+0000 7f7c73a00640 1 -- start start 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.902+0000 7f7c73a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.902+0000 7f7c73a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c73a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 0x7f7c6c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c73a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c15d1d0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c73a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c0b0240 con 0x7f7c6c15cdf0 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c73a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c15d350 con 0x7f7c6c06ce80 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41028/0 (socket says 172.21.15.106:41028) 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 -- 172.21.15.106:0/780516324 learned_addr learned my addr 172.21.15.106:0/780516324 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c71000640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c71a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 0x7f7c6c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 -- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 msgr2=0x7f7c6c0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 -- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 msgr2=0x7f7c6c15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 0x7f7c6c15f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c63e00640 1 -- 172.21.15.106:0/780516324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c15daa0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.903+0000 7f7c71a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 0x7f7c6c15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c63e00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e9e0 tx=0x7f7c5400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54017070 con 0x7f7c6c0cd650 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5400c9d0 con 0x7f7c6c0cd650 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400cd80 con 0x7f7c6c0cd650 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c0cc420 con 0x7f7c6c0cd650 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.904+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c09bf80 con 0x7f7c6c0cd650 2024-04-26T21:45:56.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.905+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c069540 con 0x7f7c6c0cd650 2024-04-26T21:45:56.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.906+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c5401a070 con 0x7f7c6c0cd650 2024-04-26T21:45:56.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.907+0000 7f7c69600640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.908+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f7c5408ba60 con 0x7f7c6c0cd650 2024-04-26T21:45:56.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.908+0000 7f7c71000640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.908+0000 7f7c71000640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 0x7f7c4806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:56.912+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c54059df0 con 0x7f7c6c0cd650 2024-04-26T21:45:57.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.113+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c6c09e860 con 0x7f7c6c0cd650 2024-04-26T21:45:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.116+0000 7f7c69600640 1 -- 172.21.15.106:0/780516324 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16630 (secure 0 0 0) 0x7f7c540596f0 con 0x7f7c6c0cd650 2024-04-26T21:45:57.117 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:57.117 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:56.102128+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 msgr2=0x7f7c4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 0x7f7c4806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c009040 comp rx=0 tx=0).stop 2024-04-26T21:45:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 msgr2=0x7f7c6c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e9e0 tx=0x7f7c5400eea0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 shutdown_connections 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6c15cdf0 0x7f7c6c15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0cd650 0x7f7c6c15d560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 --2- 172.21.15.106:0/780516324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c6c06ce80 0x7f7c6c0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 >> 172.21.15.106:0/780516324 conn(0x7f7c6c0b2d50 msgr2=0x7f7c6c0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 shutdown_connections 2024-04-26T21:45:57.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.119+0000 7f7c73a00640 1 -- 172.21.15.106:0/780516324 wait complete. 2024-04-26T21:45:57.137 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:57.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.324+0000 7feb50c00640 1 Processor -- start 2024-04-26T21:45:57.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.324+0000 7feb50c00640 1 -- start start 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb50c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0d0e70 0x7feb4c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb50c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb4c0b5660 0x7feb4c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb50c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4c0d1790 con 0x7feb4c0b5d40 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb50c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feb4c0727f0 con 0x7feb4c0b5660 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb50c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c0b4a10 con 0x7feb4c0d0e70 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb4aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb4c0b5660 0x7feb4c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb4be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feb4c0b5d40 0x7feb4c0aee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49826/0 (socket says 172.21.15.106:49826) 2024-04-26T21:45:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb4be00640 1 -- 172.21.15.106:0/444845360 learned_addr learned my addr 172.21.15.106:0/444845360 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:57.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb4b400640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0d0e70 0x7feb4c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.326+0000 7feb4a000640 1 -- 172.21.15.106:0/444845360 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391710273 0 0) 0x7feb4c0d1790 con 0x7feb4c0b5d40 2024-04-26T21:45:57.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.327+0000 7feb4aa00640 1 -- 172.21.15.106:0/444845360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0d0e70 msgr2=0x7feb4c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.327+0000 7feb4aa00640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0d0e70 0x7feb4c0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.327+0000 7feb4aa00640 1 -- 172.21.15.106:0/444845360 >> v1:172.21.15.106:6789/0 conn(0x7feb4c0b5d40 legacy=0x7feb4c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.327+0000 7feb4aa00640 1 -- 172.21.15.106:0/444845360 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c0b4cf0 con 0x7feb4c0b5660 2024-04-26T21:45:57.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb4aa00640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c0ae8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb380097a0 tx=0x7feb3802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ed3ace7bab65ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb4a000640 1 -- 172.21.15.106:0/444845360 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38037070 con 0x7feb4c0b5660 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb4a000640 1 -- 172.21.15.106:0/444845360 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3802ae50 con 0x7feb4c0b5660 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb4a000640 1 -- 172.21.15.106:0/444845360 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38032660 con 0x7feb4c0b5660 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 -- 172.21.15.106:0/444845360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 msgr2=0x7feb4c0ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c0ae8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb380097a0 tx=0x7feb3802a100 comp rx=0 tx=0).stop 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 -- 172.21.15.106:0/444845360 shutdown_connections 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c0ae8c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 --2- 172.21.15.106:0/444845360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0d0e70 0x7feb4c0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.328+0000 7feb50c00640 1 -- 172.21.15.106:0/444845360 >> 172.21.15.106:0/444845360 conn(0x7feb4c06c740 msgr2=0x7feb4c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.329+0000 7feb50c00640 1 -- 172.21.15.106:0/444845360 shutdown_connections 2024-04-26T21:45:57.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.329+0000 7feb50c00640 1 -- 172.21.15.106:0/444845360 wait complete. 2024-04-26T21:45:57.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.330+0000 7feb50c00640 1 Processor -- start 2024-04-26T21:45:57.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.330+0000 7feb50c00640 1 -- start start 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0b5d40 0x7feb4c14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb4c0d0e70 0x7feb4c161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c0b05b0 con 0x7feb4c0b5660 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c0b0430 con 0x7feb4c0d0e70 2024-04-26T21:45:57.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb50c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c15f1a0 con 0x7feb4c0b5d40 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41044/0 (socket says 172.21.15.106:41044) 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.331+0000 7feb4b400640 1 -- 172.21.15.106:0/125745703 learned_addr learned my addr 172.21.15.106:0/125745703 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 -- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0b5d40 msgr2=0x7feb4c14b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0b5d40 0x7feb4c14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 -- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb4c0d0e70 msgr2=0x7feb4c161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb4c0d0e70 0x7feb4c161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 -- 172.21.15.106:0/125745703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c15a490 con 0x7feb4c0b5660 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb4b400640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d830 tx=0x7feb3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00b6c0 con 0x7feb4c0b5660 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3c019070 con 0x7feb4c0b5660 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00ba80 con 0x7feb4c0b5660 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4c151f80 con 0x7feb4c0b5660 2024-04-26T21:45:57.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.332+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4c152540 con 0x7feb4c0b5660 2024-04-26T21:45:57.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.334+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4c14f7d0 con 0x7feb4c0b5660 2024-04-26T21:45:57.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.334+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb3c01e460 con 0x7feb4c0b5660 2024-04-26T21:45:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.335+0000 7feb43400640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 0x7feb2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.335+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7feb3c08c2d0 con 0x7feb4c0b5660 2024-04-26T21:45:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.335+0000 7feb4aa00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 0x7feb2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.336+0000 7feb4aa00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 0x7feb2c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb38034000 tx=0x7feb3802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.340+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb4c14f7d0 con 0x7feb4c0b5660 2024-04-26T21:45:57.497 INFO:tasks.ceph.mds.d.smithi204.stderr:2024-04-26T21:45:57.496+0000 7f1904000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:45:57.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.527+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb4c0693c0 con 0x7feb4c0b5660 2024-04-26T21:45:57.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.530+0000 7feb43400640 1 -- 172.21.15.106:0/125745703 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16630 (secure 0 0 0) 0x7feb3c05a660 con 0x7feb4c0b5660 2024-04-26T21:45:57.531 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:45:57.531 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:56.102128+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:57.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 msgr2=0x7feb2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 0x7feb2c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb38034000 tx=0x7feb3802c000 comp rx=0 tx=0).stop 2024-04-26T21:45:57.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 msgr2=0x7feb4c151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d830 tx=0x7feb3c00dcf0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 shutdown_connections 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb4c0d0e70 0x7feb4c161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2c06c8f0 0x7feb2c06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4c0b5d40 0x7feb4c14b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 --2- 172.21.15.106:0/125745703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb4c0b5660 0x7feb4c151810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 >> 172.21.15.106:0/125745703 conn(0x7feb4c06c740 msgr2=0x7feb4c0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 shutdown_connections 2024-04-26T21:45:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.533+0000 7feb50c00640 1 -- 172.21.15.106:0/125745703 wait complete. 2024-04-26T21:45:57.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:45:57.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.747+0000 7fcc12a00640 1 Processor -- start 2024-04-26T21:45:57.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.747+0000 7fcc12a00640 1 -- start start 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc12a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc0c15d100 0x7fcc0c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc12a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcc0c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc12a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0c0af800 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc12a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0c0ae340 con 0x7fcc0c15d100 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc12a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c0afaa0 con 0x7fcc0c0add80 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc03400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcc0c0b5ca0 0x7fcc0c072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49828/0 (socket says 172.21.15.106:49828) 2024-04-26T21:45:57.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc03400640 1 -- 172.21.15.106:0/4214119463 learned_addr learned my addr 172.21.15.106:0/4214119463 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371942352 0 0) 0x7fcc0c0af800 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbf0003520 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc10a00640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcc0c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.749+0000 7fcc0be00640 1 --2- 172.21.15.106:0/4214119463 >> v2:172.21.15.204:3300/0 conn(0x7fcc0c15d100 0x7fcc0c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42759217 0 0) 0x7fcbf0003520 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 msgr2=0x7fcc0c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcc0c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 >> v2:172.21.15.204:3300/0 conn(0x7fcc0c15d100 msgr2=0x7fcc0c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 --2- 172.21.15.106:0/4214119463 >> v2:172.21.15.204:3300/0 conn(0x7fcc0c15d100 0x7fcc0c06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c0afd80 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fcbf4001670 con 0x7fcc0c0b5ca0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 >> v1:172.21.15.106:6789/0 conn(0x7fcc0c0b5ca0 legacy=0x7fcc0c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf00039b0 0x7fcbf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbf0004800 0x7fcbf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcbf00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c0ae340 con 0x7fcbf0004800 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c0afaa0 con 0x7fcbf00039b0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf000bb50 con 0x7fcc0c0add80 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc03400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbf0004800 0x7fcbf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc0be00640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf00039b0 0x7fcbf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.750+0000 7fcc10a00640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcbf00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.751+0000 7fcc03400640 1 -- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 msgr2=0x7fcbf00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.751+0000 7fcc03400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c0add80 0x7fcbf00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.751+0000 7fcc03400640 1 -- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf00039b0 msgr2=0x7fcbf0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.751+0000 7fcc03400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf00039b0 0x7fcbf0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.751+0000 7fcc03400640 1 -- 172.21.15.106:0/4214119463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf0003520 con 0x7fcbf0004800 2024-04-26T21:45:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc03400640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbf0004800 0x7fcbf0004c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c160 tx=0x7fcbf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1006dd63a5463b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007c80 con 0x7fcbf0004800 2024-04-26T21:45:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf4005ad0 con 0x7fcbf0004800 2024-04-26T21:45:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc0b400640 1 -- 172.21.15.106:0/4214119463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007070 con 0x7fcbf0004800 2024-04-26T21:45:57.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc12a00640 1 -- 172.21.15.106:0/4214119463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbf0004800 msgr2=0x7fcbf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc12a00640 1 --2- 172.21.15.106:0/4214119463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbf0004800 0x7fcbf0004c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c160 tx=0x7fcbf0005830 comp rx=0 tx=0).stop 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc10a00640 1 -- 172.21.15.106:0/4214119463 reap_dead start 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc12a00640 1 -- 172.21.15.106:0/4214119463 shutdown_connections 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc12a00640 1 -- 172.21.15.106:0/4214119463 >> 172.21.15.106:0/4214119463 conn(0x7fcc0c0b2d50 msgr2=0x7fcc0c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.752+0000 7fcc12a00640 1 -- 172.21.15.106:0/4214119463 shutdown_connections 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.753+0000 7fcc12a00640 1 -- 172.21.15.106:0/4214119463 wait complete. 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.754+0000 7fcc12a00640 1 Processor -- start 2024-04-26T21:45:57.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.754+0000 7fcc12a00640 1 -- start start 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf0004800 0x7fcc0c1486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15b2f0 0x7fcc0c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c09ee50 con 0x7fcc0c15e510 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c0b0340 con 0x7fcc0c15b2f0 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc12a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c160b70 con 0x7fcbf0004800 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc10a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc10a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41054/0 (socket says 172.21.15.106:41054) 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.755+0000 7fcc10a00640 1 -- 172.21.15.106:0/2784893007 learned_addr learned my addr 172.21.15.106:0/2784893007 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:57.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc0be00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf0004800 0x7fcc0c1486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc03400640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15b2f0 0x7fcc0c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 -- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf0004800 msgr2=0x7fcc0c1486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf0004800 0x7fcc0c1486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 -- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15b2f0 msgr2=0x7fcc0c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15b2f0 0x7fcc0c15b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 -- 172.21.15.106:0/2784893007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c158e50 con 0x7fcc0c15e510 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc10a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400f410 tx=0x7fcc0400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004440 con 0x7fcc0c15e510 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc04019070 con 0x7fcc0c15e510 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c159130 con 0x7fcc0c15e510 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04003850 con 0x7fcc0c15e510 2024-04-26T21:45:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.756+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c148c00 con 0x7fcc0c15e510 2024-04-26T21:45:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.757+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c069650 con 0x7fcc0c15e510 2024-04-26T21:45:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.758+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc04012070 con 0x7fcc0c15e510 2024-04-26T21:45:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.759+0000 7fcc08c00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.759+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc0408b320 con 0x7fcc0c15e510 2024-04-26T21:45:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.759+0000 7fcc0be00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.760+0000 7fcc0be00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0042f0 tx=0x7fcbfc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.763+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc04059750 con 0x7fcc0c15e510 2024-04-26T21:45:57.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:57.988+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcc0c073d00 con 0x7fcc0c15e510 2024-04-26T21:45:58.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.686+0000 7fcc08c00640 1 -- 172.21.15.106:0/2784893007 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fcc04059050 con 0x7fcc0c15e510 2024-04-26T21:45:58.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 msgr2=0x7fcbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0042f0 tx=0x7fcbfc00b040 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 msgr2=0x7fcc0c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400f410 tx=0x7fcc0400f8d0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 shutdown_connections 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc0c15e510 0x7fcc0c15e990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15b2f0 0x7fcc0c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 --2- 172.21.15.106:0/2784893007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf0004800 0x7fcc0c1486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.690+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 >> 172.21.15.106:0/2784893007 conn(0x7fcc0c0b2d50 msgr2=0x7fcc0c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.691+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 shutdown_connections 2024-04-26T21:45:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.691+0000 7fcc12a00640 1 -- 172.21.15.106:0/2784893007 wait complete. 2024-04-26T21:45:58.714 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:45:58.715 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:45:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.916+0000 7f9a97200640 1 Processor -- start 2024-04-26T21:45:58.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.916+0000 7f9a97200640 1 -- start start 2024-04-26T21:45:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.917+0000 7f9a97200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 0x7f9a9806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.917+0000 7f9a97200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.917+0000 7f9a97200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a980b4d20 con 0x7f9a98160d60 2024-04-26T21:45:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.917+0000 7f9a97200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9a98146c30 con 0x7f9a98161140 2024-04-26T21:45:58.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.917+0000 7f9a97200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980b5000 con 0x7f9a98146280 2024-04-26T21:45:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.918+0000 7f9a8fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 0x7f9a9806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.918+0000 7f9a96800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.918+0000 7f9a8fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 0x7f9a9806dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37260/0 (socket says 172.21.15.106:37260) 2024-04-26T21:45:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.918+0000 7f9a8fe00640 1 -- 172.21.15.106:0/93712296 learned_addr learned my addr 172.21.15.106:0/93712296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:58.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.918+0000 7f9a96800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39284/0 (socket says 172.21.15.106:39284) 2024-04-26T21:45:58.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.919+0000 7f9a96800640 1 -- 172.21.15.106:0/93712296 >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 msgr2=0x7f9a9806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:58.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.919+0000 7f9a96800640 1 --2- 172.21.15.106:0/93712296 >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 0x7f9a9806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.919+0000 7f9a96800640 1 -- 172.21.15.106:0/93712296 >> v1:172.21.15.204:6789/0 conn(0x7f9a98160d60 legacy=0x7f9a980b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.919+0000 7f9a96800640 1 -- 172.21.15.106:0/93712296 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a980b52e0 con 0x7f9a98146280 2024-04-26T21:45:58.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.920+0000 7f9a96800640 1 --2- 172.21.15.106:0/93712296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009dc0 tx=0x7f9a9002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d321036e6c4ff27f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.920+0000 7f9a95400640 1 -- 172.21.15.106:0/93712296 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9002ae60 con 0x7f9a98146280 2024-04-26T21:45:58.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.920+0000 7f9a95400640 1 -- 172.21.15.106:0/93712296 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90007740 con 0x7f9a98146280 2024-04-26T21:45:58.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a95400640 1 -- 172.21.15.106:0/93712296 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90007b10 con 0x7f9a98146280 2024-04-26T21:45:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 -- 172.21.15.106:0/93712296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 msgr2=0x7f9a981466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 --2- 172.21.15.106:0/93712296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009dc0 tx=0x7f9a9002a680 comp rx=0 tx=0).stop 2024-04-26T21:45:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 -- 172.21.15.106:0/93712296 shutdown_connections 2024-04-26T21:45:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 --2- 172.21.15.106:0/93712296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98146280 0x7f9a981466f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 --2- 172.21.15.106:0/93712296 >> v2:172.21.15.204:3300/0 conn(0x7f9a98161140 0x7f9a9806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 -- 172.21.15.106:0/93712296 >> 172.21.15.106:0/93712296 conn(0x7f9a980b2900 msgr2=0x7f9a980b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 -- 172.21.15.106:0/93712296 shutdown_connections 2024-04-26T21:45:58.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.921+0000 7f9a97200640 1 -- 172.21.15.106:0/93712296 wait complete. 2024-04-26T21:45:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.923+0000 7f9a97200640 1 Processor -- start 2024-04-26T21:45:58.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.923+0000 7f9a97200640 1 -- start start 2024-04-26T21:45:58.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a98146280 0x7f9a9813f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98161140 0x7f9a98158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9806e420 con 0x7f9a98160d60 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980cf6f0 con 0x7f9a98146280 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a97200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9806e5a0 con 0x7f9a98161140 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51736/0 (socket says 172.21.15.106:51736) 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a8fe00640 1 -- 172.21.15.106:0/1744333166 learned_addr learned my addr 172.21.15.106:0/1744333166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a96800640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98161140 0x7f9a98158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.924+0000 7f9a95e00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a98146280 0x7f9a9813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 -- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98161140 msgr2=0x7f9a98158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98161140 0x7f9a98158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 -- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a98146280 msgr2=0x7f9a9813f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a98146280 0x7f9a9813f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 -- 172.21.15.106:0/1744333166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9813ec70 con 0x7f9a98160d60 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8fe00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e840 tx=0x7f9a8000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80017070 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8000c8e0 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9812eae0 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.925+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000ccd0 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.926+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9812f0a0 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.927+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a980cee70 con 0x7f9a98160d60 2024-04-26T21:45:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.928+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a8001a070 con 0x7f9a98160d60 2024-04-26T21:45:58.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.928+0000 7f9a8e000640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 0x7f9a7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.929+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9a8008be50 con 0x7f9a98160d60 2024-04-26T21:45:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.929+0000 7f9a95e00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 0x7f9a7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.929+0000 7f9a95e00640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 0x7f9a7406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9a88001640 tx=0x7f9a88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:58.933+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a980cee70 con 0x7f9a98160d60 2024-04-26T21:45:59.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.158+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9a9812e850 con 0x7f9a98160d60 2024-04-26T21:45:59.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.709+0000 7f9a8e000640 1 -- 172.21.15.106:0/1744333166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f9a8005a1e0 con 0x7f9a98160d60 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.713+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 msgr2=0x7f9a7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.713+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 0x7f9a7406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9a88001640 tx=0x7f9a88009040 comp rx=0 tx=0).stop 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.713+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 msgr2=0x7f9a9813e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.713+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e840 tx=0x7f9a8000ed00 comp rx=0 tx=0).stop 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 shutdown_connections 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9a7406c8f0 0x7f9a7406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a98161140 0x7f9a98158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a98160d60 0x7f9a9813e730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 --2- 172.21.15.106:0/1744333166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a98146280 0x7f9a9813f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 >> 172.21.15.106:0/1744333166 conn(0x7f9a980b2900 msgr2=0x7f9a98145780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:59.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 shutdown_connections 2024-04-26T21:45:59.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.714+0000 7f9a97200640 1 -- 172.21.15.106:0/1744333166 wait complete. 2024-04-26T21:45:59.736 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:45:59.736 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.935+0000 7f1dfa600640 1 Processor -- start 2024-04-26T21:45:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.935+0000 7f1dfa600640 1 -- start start 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1dfa600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1df406ce80 0x7f1df40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1dfa600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1df40cd650 0x7f1df40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1dfa600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df40af9a0 con 0x7f1df415cdf0 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1dfa600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1df40ae2c0 con 0x7f1df406ce80 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1dfa600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1df40b05b0 con 0x7f1df40cd650 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1df406ce80 0x7f1df40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1df406ce80 0x7f1df40add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51752/0 (socket says 172.21.15.106:51752) 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 -- 172.21.15.106:0/1260902446 learned_addr learned my addr 172.21.15.106:0/1260902446 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:59.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df3e00640 1 --2- 172.21.15.106:0/1260902446 >> v2:172.21.15.204:3300/0 conn(0x7f1df40cd650 0x7f1df40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 -- 172.21.15.106:0/1260902446 >> v2:172.21.15.204:3300/0 conn(0x7f1df40cd650 msgr2=0x7f1df40b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 --2- 172.21.15.106:0/1260902446 >> v2:172.21.15.204:3300/0 conn(0x7f1df40cd650 0x7f1df40b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.937+0000 7f1df2a00640 1 -- 172.21.15.106:0/1260902446 >> v1:172.21.15.204:6789/0 conn(0x7f1df415cdf0 legacy=0x7f1df406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.938+0000 7f1df2a00640 1 -- 172.21.15.106:0/1260902446 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df40afc80 con 0x7f1df406ce80 2024-04-26T21:45:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1df2a00640 1 --2- 172.21.15.106:0/1260902446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df406ce80 0x7f1df40add80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1df40b0520 tx=0x7f1de002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc954a2e2a0e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1df2000640 1 -- 172.21.15.106:0/1260902446 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0037070 con 0x7f1df406ce80 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1df2000640 1 -- 172.21.15.106:0/1260902446 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de0032440 con 0x7f1df406ce80 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1df2000640 1 -- 172.21.15.106:0/1260902446 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0032810 con 0x7f1df406ce80 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1dfa600640 1 -- 172.21.15.106:0/1260902446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df406ce80 msgr2=0x7f1df40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.939+0000 7f1dfa600640 1 --2- 172.21.15.106:0/1260902446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df406ce80 0x7f1df40add80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1df40b0520 tx=0x7f1de002a230 comp rx=0 tx=0).stop 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 -- 172.21.15.106:0/1260902446 shutdown_connections 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 --2- 172.21.15.106:0/1260902446 >> v2:172.21.15.204:3300/0 conn(0x7f1df40cd650 0x7f1df40b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 --2- 172.21.15.106:0/1260902446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df406ce80 0x7f1df40add80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 -- 172.21.15.106:0/1260902446 >> 172.21.15.106:0/1260902446 conn(0x7f1df40b2d50 msgr2=0x7f1df40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 -- 172.21.15.106:0/1260902446 shutdown_connections 2024-04-26T21:45:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.940+0000 7f1dfa600640 1 -- 172.21.15.106:0/1260902446 wait complete. 2024-04-26T21:45:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.941+0000 7f1dfa600640 1 Processor -- start 2024-04-26T21:45:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.941+0000 7f1dfa600640 1 -- start start 2024-04-26T21:45:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df406ce80 0x7f1df414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df415cdf0 0x7f1df415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1df415d1d0 con 0x7f1df40cd650 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1df40b0240 con 0x7f1df415cdf0 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.942+0000 7f1dfa600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1df415d350 con 0x7f1df406ce80 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51768/0 (socket says 172.21.15.106:51768) 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 -- 172.21.15.106:0/2508783368 learned_addr learned my addr 172.21.15.106:0/2508783368 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 -- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df406ce80 msgr2=0x7f1df414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df3400640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df406ce80 0x7f1df414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df406ce80 0x7f1df414c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df3e00640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df415cdf0 0x7f1df415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 -- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df415cdf0 msgr2=0x7f1df415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df415cdf0 0x7f1df415fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 -- 172.21.15.106:0/2508783368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df409d510 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.943+0000 7f1df2a00640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000c00 tx=0x7f1de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.944+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0041020 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.944+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de00306c0 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.944+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df414cce0 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.944+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0037040 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.944+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df415e3c0 con 0x7f1df40cd650 2024-04-26T21:45:59.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.945+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df4069540 con 0x7f1df40cd650 2024-04-26T21:45:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.946+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1de0044020 con 0x7f1df40cd650 2024-04-26T21:45:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.947+0000 7f1deb400640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.947+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1de00ac0f0 con 0x7f1df40cd650 2024-04-26T21:45:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.947+0000 7f1df3400640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.948+0000 7f1df3400640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1de400a750 tx=0x7f1de4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:45:59.951+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1de004b070 con 0x7f1df40cd650 2024-04-26T21:46:00.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.191+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1df40b22c0 con 0x7f1df40cd650 2024-04-26T21:46:00.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.192+0000 7f1deb400640 1 -- 172.21.15.106:0/2508783368 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f1de007a480 con 0x7f1df40cd650 2024-04-26T21:46:00.195 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:00.196 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 msgr2=0x7f1dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1de400a750 tx=0x7f1de4008040 comp rx=0 tx=0).stop 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 msgr2=0x7f1df41f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000c00 tx=0x7f1de00046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 shutdown_connections 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df415cdf0 0x7f1df415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df40cd650 0x7f1df41f2a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 --2- 172.21.15.106:0/2508783368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df406ce80 0x7f1df414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 >> 172.21.15.106:0/2508783368 conn(0x7f1df40b2d50 msgr2=0x7f1df406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:00.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 shutdown_connections 2024-04-26T21:46:00.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.195+0000 7f1dfa600640 1 -- 172.21.15.106:0/2508783368 wait complete. 2024-04-26T21:46:00.215 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.412+0000 7f324fe00640 1 Processor -- start 2024-04-26T21:46:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.412+0000 7f324fe00640 1 -- start start 2024-04-26T21:46:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.414+0000 7f324fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32480cd650 0x7f32480add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.414+0000 7f324fe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32480af9a0 con 0x7f324815cdf0 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.414+0000 7f324fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32480afc80 con 0x7f324806ce80 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.414+0000 7f324fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f32480ae2c0 con 0x7f32480cd650 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.414+0000 7f324d400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f324815cdf0 0x7f324806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58764/0 (socket says 172.21.15.106:58764) 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f324d400640 1 -- 172.21.15.106:0/3043801997 learned_addr learned my addr 172.21.15.106:0/3043801997 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f324de00640 1 --2- 172.21.15.106:0/3043801997 >> v2:172.21.15.204:3300/0 conn(0x7f32480cd650 0x7f32480add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847865397 0 0) 0x7f32480af9a0 con 0x7f324815cdf0 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3230003520 con 0x7f324815cdf0 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2100192666 0 0) 0x7f3230003520 con 0x7f324815cdf0 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 >> v2:172.21.15.204:3300/0 conn(0x7f32480cd650 msgr2=0x7f32480add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 --2- 172.21.15.106:0/3043801997 >> v2:172.21.15.204:3300/0 conn(0x7f32480cd650 0x7f32480add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 >> v1:172.21.15.204:6789/0 conn(0x7f324806ce80 legacy=0x7f32480b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32480aff60 con 0x7f324815cdf0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.415+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f323c0019f0 con 0x7f324815cdf0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 >> v1:172.21.15.106:6789/0 conn(0x7f324815cdf0 legacy=0x7f324806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f3230003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3230004800 0x7f3230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32300051c0 0x7f323000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f323000c090 con 0x7f3230003a00 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f323000c240 con 0x7f3230004800 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f323000c440 con 0x7f32300051c0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f3230003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324ca00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3230004800 0x7f3230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324de00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32300051c0 0x7f323000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 -- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32300051c0 msgr2=0x7f323000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32300051c0 0x7f323000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 -- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3230004800 msgr2=0x7f3230004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3230004800 0x7f3230004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324d400640 1 -- 172.21.15.106:0/3043801997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3230003520 con 0x7f3230003a00 2024-04-26T21:46:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.416+0000 7f324de00640 1 -- 172.21.15.106:0/3043801997 reap_dead start 2024-04-26T21:46:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.417+0000 7f324d400640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f3230003e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f323c00a770 tx=0x7f3230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9225f4d7acaae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00a390 con 0x7f3230003a00 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c02f070 con 0x7f3230003a00 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f3247e00640 1 -- 172.21.15.106:0/3043801997 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c02bbd0 con 0x7f3230003a00 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 -- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 msgr2=0x7f3230003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f3230003e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f323c00a770 tx=0x7f3230005830 comp rx=0 tx=0).stop 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 -- 172.21.15.106:0/3043801997 shutdown_connections 2024-04-26T21:46:00.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 --2- 172.21.15.106:0/3043801997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f3230003e20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 -- 172.21.15.106:0/3043801997 >> 172.21.15.106:0/3043801997 conn(0x7f32480b2d50 msgr2=0x7f32480b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:00.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 -- 172.21.15.106:0/3043801997 shutdown_connections 2024-04-26T21:46:00.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.418+0000 7f324fe00640 1 -- 172.21.15.106:0/3043801997 wait complete. 2024-04-26T21:46:00.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.420+0000 7f324fe00640 1 Processor -- start 2024-04-26T21:46:00.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.420+0000 7f324fe00640 1 -- start start 2024-04-26T21:46:00.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f324809cc30 0x7f324815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324809d0b0 0x7f32480b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f324815d1d0 con 0x7f3230003a00 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32480b05b0 con 0x7f324809d0b0 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.421+0000 7f324fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f324815d350 con 0x7f324809cc30 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51790/0 (socket says 172.21.15.106:51790) 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 -- 172.21.15.106:0/659257572 learned_addr learned my addr 172.21.15.106:0/659257572 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324ca00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f324809cc30 0x7f324815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324de00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324809d0b0 0x7f32480b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 -- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f324809cc30 msgr2=0x7f324815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f324809cc30 0x7f324815fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 -- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324809d0b0 msgr2=0x7f32480b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324809d0b0 0x7f32480b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 -- 172.21.15.106:0/659257572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324809bd30 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.422+0000 7f324d400640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f323c00a770 tx=0x7f32480afe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.423+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c001250 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.423+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c034470 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.423+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c034840 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.423+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32481f8320 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.423+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32481f88e0 con 0x7f3230003a00 2024-04-26T21:46:00.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.424+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3248069540 con 0x7f3230003a00 2024-04-26T21:46:00.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.425+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f323c047060 con 0x7f3230003a00 2024-04-26T21:46:00.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.426+0000 7f3245600640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 0x7f322806efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.426+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f323c0abaa0 con 0x7f3230003a00 2024-04-26T21:46:00.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.426+0000 7f324ca00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 0x7f322806efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.427+0000 7f324ca00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 0x7f322806efa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f32480b5d60 tx=0x7f3234009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.430+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f323c048e90 con 0x7f3230003a00 2024-04-26T21:46:00.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.634+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3248069540 con 0x7f3230003a00 2024-04-26T21:46:00.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.636+0000 7f3245600640 1 -- 172.21.15.106:0/659257572 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f323c07a040 con 0x7f3230003a00 2024-04-26T21:46:00.636 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:00.637 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 msgr2=0x7f322806efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 0x7f322806efa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f32480b5d60 tx=0x7f3234009040 comp rx=0 tx=0).stop 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 msgr2=0x7f32481571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f323c00a770 tx=0x7f32480afe50 comp rx=0 tx=0).stop 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 shutdown_connections 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324809d0b0 0x7f32480b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f322806cae0 0x7f322806efa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f324809cc30 0x7f324815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 --2- 172.21.15.106:0/659257572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3230003a00 0x7f32481571c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 >> 172.21.15.106:0/659257572 conn(0x7f32480b2d50 msgr2=0x7f324815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 shutdown_connections 2024-04-26T21:46:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.639+0000 7f324fe00640 1 -- 172.21.15.106:0/659257572 wait complete. 2024-04-26T21:46:00.658 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:46:00.658 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:46:00.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.848+0000 7f940e000640 1 Processor -- start 2024-04-26T21:46:00.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.848+0000 7f940e000640 1 -- start start 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f940e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f94080add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f940e000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94080af9a0 con 0x7f94080cd650 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f940e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94080afc80 con 0x7f940806ce80 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f940e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f94080ae2c0 con 0x7f940815cdf0 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f9406a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f94080cd650 0x7f940806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58780/0 (socket says 172.21.15.106:58780) 2024-04-26T21:46:00.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f9406a00640 1 -- 172.21.15.106:0/1004405325 learned_addr learned my addr 172.21.15.106:0/1004405325 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.850+0000 7f9407400640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f94080add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124365635 0 0) 0x7f94080af9a0 con 0x7f94080cd650 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93ec003520 con 0x7f94080cd650 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1119097430 0 0) 0x7f93ec003520 con 0x7f94080cd650 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 msgr2=0x7f94080add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f94080add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 >> v1:172.21.15.204:6789/0 conn(0x7f940806ce80 legacy=0x7f94080b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94080aff60 con 0x7f94080cd650 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f93f4001670 con 0x7f94080cd650 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 >> v1:172.21.15.106:6789/0 conn(0x7f94080cd650 legacy=0x7f940806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93ec0039e0 0x7f93ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f93ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec00ba70 con 0x7f93ec004800 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec00bbf0 con 0x7f93ec0039e0 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec00bdf0 con 0x7f940815cdf0 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.851+0000 7f9406a00640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9407e00640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f93ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9407400640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93ec0039e0 0x7f93ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9406a00640 1 -- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 msgr2=0x7f93ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9406a00640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815cdf0 0x7f93ec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9406a00640 1 -- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93ec0039e0 msgr2=0x7f93ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9406a00640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93ec0039e0 0x7f93ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9406a00640 1 -- 172.21.15.106:0/1004405325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec003520 con 0x7f93ec004800 2024-04-26T21:46:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.852+0000 7f9407400640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93ec0039e0 0x7f93ec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.853+0000 7f9406a00640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c210 tx=0x7f93ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c52c2b6e9a5851 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.853+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4007cf0 con 0x7f93ec004800 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.853+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f4005af0 con 0x7f93ec004800 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.853+0000 7f9406000640 1 -- 172.21.15.106:0/1004405325 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4007070 con 0x7f93ec004800 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 -- 172.21.15.106:0/1004405325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 msgr2=0x7f93ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 --2- 172.21.15.106:0/1004405325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c210 tx=0x7f93ec005830 comp rx=0 tx=0).stop 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f9407e00640 1 -- 172.21.15.106:0/1004405325 reap_dead start 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 -- 172.21.15.106:0/1004405325 shutdown_connections 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 -- 172.21.15.106:0/1004405325 >> 172.21.15.106:0/1004405325 conn(0x7f94080b2d50 msgr2=0x7f94080b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 -- 172.21.15.106:0/1004405325 shutdown_connections 2024-04-26T21:46:00.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.854+0000 7f940e000640 1 -- 172.21.15.106:0/1004405325 wait complete. 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.856+0000 7f940e000640 1 Processor -- start 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.856+0000 7f940e000640 1 -- start start 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.856+0000 7f940e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f940e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940806f570 0x7f940806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f940e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815f400 0x7f940815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f940e000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f940815d1d0 con 0x7f93ec004800 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f940e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94080b05b0 con 0x7f940806f570 2024-04-26T21:46:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f940e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f940815d350 con 0x7f940815f400 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51816/0 (socket says 172.21.15.106:51816) 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 -- 172.21.15.106:0/4175172279 learned_addr learned my addr 172.21.15.106:0/4175172279 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 -- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815f400 msgr2=0x7f940815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815f400 0x7f940815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 -- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940806f570 msgr2=0x7f940806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940806f570 0x7f940806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 -- 172.21.15.106:0/4175172279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94080cc9b0 con 0x7f93ec004800 2024-04-26T21:46:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.857+0000 7f9407400640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93f80108c0 tx=0x7f93f8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.858+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800f040 con 0x7f93ec004800 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.858+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f800eb00 con 0x7f93ec004800 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.858+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f801c400 con 0x7f93ec004800 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.858+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9408160230 con 0x7f93ec004800 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.858+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940809c430 con 0x7f93ec004800 2024-04-26T21:46:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.859+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940809bf40 con 0x7f93ec004800 2024-04-26T21:46:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.860+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93f801c5a0 con 0x7f93ec004800 2024-04-26T21:46:00.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.861+0000 7f93ff400640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:00.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.861+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f93f808c210 con 0x7f93ec004800 2024-04-26T21:46:00.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.861+0000 7f9406a00640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.862+0000 7f9406a00640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0057e0 tx=0x7f93f4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:00.865+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93f8018030 con 0x7f93ec004800 2024-04-26T21:46:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.065+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9408069540 con 0x7f93e806c8f0 2024-04-26T21:46:01.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.074+0000 7f93ff400640 1 -- 172.21.15.106:0/4175172279 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7f9408069540 con 0x7f93e806c8f0 2024-04-26T21:46:01.078 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:46:01.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 msgr2=0x7f93e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0057e0 tx=0x7f93f4007000 comp rx=0 tx=0).stop 2024-04-26T21:46:01.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 msgr2=0x7f94080cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93f80108c0 tx=0x7f93f8010d80 comp rx=0 tx=0).stop 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 shutdown_connections 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f940815f400 0x7f940815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940806f570 0x7f940806f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 --2- 172.21.15.106:0/4175172279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93ec004800 0x7f94080cc470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.080+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 >> 172.21.15.106:0/4175172279 conn(0x7f94080b2d50 msgr2=0x7f94080cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.081+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 shutdown_connections 2024-04-26T21:46:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.081+0000 7f940e000640 1 -- 172.21.15.106:0/4175172279 wait complete. 2024-04-26T21:46:01.099 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T21:45:59.088904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218744,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755976,"statfs":{"total":773094113280,"available":772870119424,"internally_reserved":0,"allocated":2519040,"data_stored":1587591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.456328"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.637342+0000","last_change":"2024-04-26T21:45:54.637342+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.637342+0000","last_undegraded":"2024-04-26T21:45:54.637342+0000","last_fullsized":"2024-04-26T21:45:54.637342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.647625+0000","last_change":"2024-04-26T21:45:54.647625+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.647625+0000","last_undegraded":"2024-04-26T21:45:54.647625+0000","last_fullsized":"2024-04-26T21:45:54.647625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.638190+0000","last_change":"2024-04-26T21:45:54.638190+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.638190+0000","last_undegraded":"2024-04-26T21:45:54.638190+0000","last_fullsized":"2024-04-26T21:45:54.638190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.651246+0000","last_change":"2024-04-26T21:45:54.651246+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.651246+0000","last_undegraded":"2024-04-26T21:45:54.651246+0000","last_fullsized":"2024-04-26T21:45:54.651246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.649302+0000","last_change":"2024-04-26T21:45:54.649302+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.649302+0000","last_undegraded":"2024-04-26T21:45:54.649302+0000","last_fullsized":"2024-04-26T21:45:54.649302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.653582+0000","last_change":"2024-04-26T21:45:54.653582+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.653582+0000","last_undegraded":"2024-04-26T21:45:54.653582+0000","last_fullsized":"2024-04-26T21:45:54.653582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.645652+0000","last_change":"2024-04-26T21:45:54.645652+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.645652+0000","last_undegraded":"2024-04-26T21:45:54.645652+0000","last_fullsized":"2024-04-26T21:45:54.645652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.646197+0000","last_change":"2024-04-26T21:45:54.646197+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.646197+0000","last_undegraded":"2024-04-26T21:45:54.646197+0000","last_fullsized":"2024-04-26T21:45:54.646197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.645744+0000","last_change":"2024-04-26T21:45:54.645744+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.645744+0000","last_undegraded":"2024-04-26T21:45:54.645744+0000","last_fullsized":"2024-04-26T21:45:54.645744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.655434+0000","last_change":"2024-04-26T21:45:54.655434+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.655434+0000","last_undegraded":"2024-04-26T21:45:54.655434+0000","last_fullsized":"2024-04-26T21:45:54.655434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.646078+0000","last_change":"2024-04-26T21:45:54.646078+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.646078+0000","last_undegraded":"2024-04-26T21:45:54.646078+0000","last_fullsized":"2024-04-26T21:45:54.646078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.658423+0000","last_change":"2024-04-26T21:45:54.658423+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.658423+0000","last_undegraded":"2024-04-26T21:45:54.658423+0000","last_fullsized":"2024-04-26T21:45:54.658423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.640653+0000","last_change":"2024-04-26T21:45:54.640653+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.640653+0000","last_undegraded":"2024-04-26T21:45:54.640653+0000","last_fullsized":"2024-04-26T21:45:54.640653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.659326+0000","last_change":"2024-04-26T21:45:54.659326+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.659326+0000","last_undegraded":"2024-04-26T21:45:54.659326+0000","last_fullsized":"2024-04-26T21:45:54.659326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.644323+0000","last_change":"2024-04-26T21:45:54.644323+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.644323+0000","last_undegraded":"2024-04-26T21:45:54.644323+0000","last_fullsized":"2024-04-26T21:45:54.644323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.658298+0000","last_change":"2024-04-26T21:45:54.658298+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.658298+0000","last_undegraded":"2024-04-26T21:45:54.658298+0000","last_fullsized":"2024-04-26T21:45:54.658298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T21:45:54.648213+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:45:54.648213+0000","last_peered":"2024-04-26T21:45:54.648213+0000","last_clean":"2024-04-26T21:45:54.648213+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:45:54.648213+0000","last_undegraded":"2024-04-26T21:45:54.648213+0000","last_fullsized":"2024-04-26T21:45:54.648213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.653906+0000","last_change":"2024-04-26T21:45:54.653906+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.653906+0000","last_undegraded":"2024-04-26T21:45:54.653906+0000","last_fullsized":"2024-04-26T21:45:54.653906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.638649+0000","last_change":"2024-04-26T21:45:54.638649+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.638649+0000","last_undegraded":"2024-04-26T21:45:54.638649+0000","last_fullsized":"2024-04-26T21:45:54.638649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.658574+0000","last_change":"2024-04-26T21:45:54.658574+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.658574+0000","last_undegraded":"2024-04-26T21:45:54.658574+0000","last_fullsized":"2024-04-26T21:45:54.658574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.651464+0000","last_change":"2024-04-26T21:45:54.651464+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.651464+0000","last_undegraded":"2024-04-26T21:45:54.651464+0000","last_fullsized":"2024-04-26T21:45:54.651464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.644221+0000","last_change":"2024-04-26T21:45:54.644221+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.644221+0000","last_undegraded":"2024-04-26T21:45:54.644221+0000","last_fullsized":"2024-04-26T21:45:54.644221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.638923+0000","last_change":"2024-04-26T21:45:54.638923+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.638923+0000","last_undegraded":"2024-04-26T21:45:54.638923+0000","last_fullsized":"2024-04-26T21:45:54.638923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.644851+0000","last_change":"2024-04-26T21:45:54.644851+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.644851+0000","last_undegraded":"2024-04-26T21:45:54.644851+0000","last_fullsized":"2024-04-26T21:45:54.644851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.647108+0000","last_change":"2024-04-26T21:45:54.647108+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.647108+0000","last_undegraded":"2024-04-26T21:45:54.647108+0000","last_fullsized":"2024-04-26T21:45:54.647108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.641413+0000","last_change":"2024-04-26T21:45:54.641413+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.641413+0000","last_undegraded":"2024-04-26T21:45:54.641413+0000","last_fullsized":"2024-04-26T21:45:54.641413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.644003+0000","last_change":"2024-04-26T21:45:54.644003+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.644003+0000","last_undegraded":"2024-04-26T21:45:54.644003+0000","last_fullsized":"2024-04-26T21:45:54.644003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.643884+0000","last_change":"2024-04-26T21:45:54.643884+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.643884+0000","last_undegraded":"2024-04-26T21:45:54.643884+0000","last_fullsized":"2024-04-26T21:45:54.643884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.638983+0000","last_change":"2024-04-26T21:45:54.638983+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.638983+0000","last_undegraded":"2024-04-26T21:45:54.638983+0000","last_fullsized":"2024-04-26T21:45:54.638983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.640588+0000","last_change":"2024-04-26T21:45:54.640588+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.640588+0000","last_undegraded":"2024-04-26T21:45:54.640588+0000","last_fullsized":"2024-04-26T21:45:54.640588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.643058+0000","last_change":"2024-04-26T21:45:54.643058+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.643058+0000","last_undegraded":"2024-04-26T21:45:54.643058+0000","last_fullsized":"2024-04-26T21:45:54.643058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.637852+0000","last_change":"2024-04-26T21:45:54.637852+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.637852+0000","last_undegraded":"2024-04-26T21:45:54.637852+0000","last_fullsized":"2024-04-26T21:45:54.637852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:45:55.629861+0000","last_change":"2024-04-26T21:45:55.629861+0000","last_active":"2024-04-26T21:45:55.629861+0000","last_peered":"2024-04-26T21:45:55.629861+0000","last_clean":"2024-04-26T21:45:55.629861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:55.629861+0000","last_undegraded":"2024-04-26T21:45:55.629861+0000","last_fullsized":"2024-04-26T21:45:55.629861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T21:45:54.646358+0000","last_change":"2024-04-26T21:45:54.646358+0000","last_active":"2024-04-26T21:45:54.624511+0000","last_peered":"2024-04-26T21:45:54.624511+0000","last_clean":"2024-04-26T21:45:54.624511+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:45:54.646358+0000","last_undegraded":"2024-04-26T21:45:54.646358+0000","last_fullsized":"2024-04-26T21:45:54.646358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:46:01.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:46:01.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.337+0000 7ff561800640 1 Processor -- start 2024-04-26T21:46:01.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.338+0000 7ff561800640 1 -- start start 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff561800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff55c15ef60 0x7ff55c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff561800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff55c0d5d90 0x7ff55c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff561800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff55c0b5d30 con 0x7ff55c0d34a0 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff561800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff55c0b6010 con 0x7ff55c15ef60 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff561800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff55c0b62f0 con 0x7ff55c0d5d90 2024-04-26T21:46:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff55b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff55c0d34a0 0x7ff55c162db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58784/0 (socket says 172.21.15.106:58784) 2024-04-26T21:46:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff55b400640 1 -- 172.21.15.106:0/266485034 learned_addr learned my addr 172.21.15.106:0/266485034 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.339+0000 7ff55aa00640 1 --2- 172.21.15.106:0/266485034 >> v2:172.21.15.106:3300/0 conn(0x7ff55c15ef60 0x7ff55c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff55a000640 1 --2- 172.21.15.106:0/266485034 >> v2:172.21.15.204:3300/0 conn(0x7ff55c0d5d90 0x7ff55c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff55aa00640 1 -- 172.21.15.106:0/266485034 >> v2:172.21.15.204:3300/0 conn(0x7ff55c0d5d90 msgr2=0x7ff55c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff55aa00640 1 --2- 172.21.15.106:0/266485034 >> v2:172.21.15.204:3300/0 conn(0x7ff55c0d5d90 0x7ff55c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff55aa00640 1 -- 172.21.15.106:0/266485034 >> v1:172.21.15.106:6789/0 conn(0x7ff55c0d34a0 legacy=0x7ff55c162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff55aa00640 1 -- 172.21.15.106:0/266485034 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c0b65d0 con 0x7ff55c15ef60 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.340+0000 7ff559600640 1 -- 172.21.15.106:0/266485034 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3995406505 0 0) 0x7ff55c0b5d30 con 0x7ff55c0d34a0 2024-04-26T21:46:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.341+0000 7ff55aa00640 1 --2- 172.21.15.106:0/266485034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c15ef60 0x7ff55c15f340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d90 tx=0x7ff54c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdd751c4af2e601 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.341+0000 7ff559600640 1 -- 172.21.15.106:0/266485034 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c007740 con 0x7ff55c15ef60 2024-04-26T21:46:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.341+0000 7ff559600640 1 -- 172.21.15.106:0/266485034 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54c0078e0 con 0x7ff55c15ef60 2024-04-26T21:46:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.341+0000 7ff559600640 1 -- 172.21.15.106:0/266485034 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c007cb0 con 0x7ff55c15ef60 2024-04-26T21:46:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 -- 172.21.15.106:0/266485034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c15ef60 msgr2=0x7ff55c15f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 --2- 172.21.15.106:0/266485034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c15ef60 0x7ff55c15f340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d90 tx=0x7ff54c02a730 comp rx=0 tx=0).stop 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 -- 172.21.15.106:0/266485034 shutdown_connections 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 --2- 172.21.15.106:0/266485034 >> v2:172.21.15.204:3300/0 conn(0x7ff55c0d5d90 0x7ff55c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 --2- 172.21.15.106:0/266485034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c15ef60 0x7ff55c15f340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 -- 172.21.15.106:0/266485034 >> 172.21.15.106:0/266485034 conn(0x7ff55c09e3b0 msgr2=0x7ff55c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 -- 172.21.15.106:0/266485034 shutdown_connections 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.342+0000 7ff561800640 1 -- 172.21.15.106:0/266485034 wait complete. 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.344+0000 7ff561800640 1 Processor -- start 2024-04-26T21:46:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.344+0000 7ff561800640 1 -- start start 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d5d90 0x7ff55c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 0x7ff55c1f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c0b5b10 con 0x7ff55c0d34a0 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c0b5990 con 0x7ff55c0d5d90 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff561800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c0d11e0 con 0x7ff55c15ef60 2024-04-26T21:46:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51836/0 (socket says 172.21.15.106:51836) 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55aa00640 1 -- 172.21.15.106:0/3113623443 learned_addr learned my addr 172.21.15.106:0/3113623443 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55a000640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d5d90 0x7ff55c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55b400640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 0x7ff55c1f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.345+0000 7ff55aa00640 1 -- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 msgr2=0x7ff55c1f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55aa00640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 0x7ff55c1f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55aa00640 1 -- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d5d90 msgr2=0x7ff55c1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55aa00640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d5d90 0x7ff55c1f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55aa00640 1 -- 172.21.15.106:0/3113623443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c1f59e0 con 0x7ff55c0d34a0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55b400640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 0x7ff55c1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff55aa00640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d90 tx=0x7ff54c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c040070 con 0x7ff55c0d34a0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54c031c40 con 0x7ff55c0d34a0 2024-04-26T21:46:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c03c420 con 0x7ff55c0d34a0 2024-04-26T21:46:01.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c1f8490 con 0x7ff55c0d34a0 2024-04-26T21:46:01.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.346+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c06d5f0 con 0x7ff55c0d34a0 2024-04-26T21:46:01.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.348+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c069540 con 0x7ff55c0d34a0 2024-04-26T21:46:01.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.349+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff54c039040 con 0x7ff55c0d34a0 2024-04-26T21:46:01.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.349+0000 7ff552a00640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 0x7ff53806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.350+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff54c0ad0c0 con 0x7ff55c0d34a0 2024-04-26T21:46:01.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.350+0000 7ff55a000640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 0x7ff53806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.350+0000 7ff55a000640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 0x7ff53806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff55c1628b0 tx=0x7ff544008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.354+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff54c07b4e0 con 0x7ff55c0d34a0 2024-04-26T21:46:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.548+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff55c08d090 con 0x7ff53806c8f0 2024-04-26T21:46:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.555+0000 7ff552a00640 1 -- 172.21.15.106:0/3113623443 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7ff55c08d090 con 0x7ff53806c8f0 2024-04-26T21:46:01.558 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 msgr2=0x7ff53806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 0x7ff53806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff55c1628b0 tx=0x7ff544008040 comp rx=0 tx=0).stop 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 msgr2=0x7ff55c06d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d90 tx=0x7ff54c004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 shutdown_connections 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff53806c8f0 0x7ff53806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15ef60 0x7ff55c1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d5d90 0x7ff55c1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 --2- 172.21.15.106:0/3113623443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55c0d34a0 0x7ff55c06d0b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 >> 172.21.15.106:0/3113623443 conn(0x7ff55c09e3b0 msgr2=0x7ff55c06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.560+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 shutdown_connections 2024-04-26T21:46:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.561+0000 7ff561800640 1 -- 172.21.15.106:0/3113623443 wait complete. 2024-04-26T21:46:01.578 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T21:46:01.089541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001722"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698213+0000","last_change":"2024-04-26T21:45:56.703198+0000","last_active":"2024-04-26T21:45:57.698213+0000","last_peered":"2024-04-26T21:45:57.698213+0000","last_clean":"2024-04-26T21:45:57.698213+0000","last_became_active":"2024-04-26T21:45:56.700438+0000","last_became_peered":"2024-04-26T21:45:56.700438+0000","last_unstale":"2024-04-26T21:45:57.698213+0000","last_undegraded":"2024-04-26T21:45:57.698213+0000","last_fullsized":"2024-04-26T21:45:57.698213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:44.744042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688536+0000","last_change":"2024-04-26T21:45:56.699815+0000","last_active":"2024-04-26T21:45:57.688536+0000","last_peered":"2024-04-26T21:45:57.688536+0000","last_clean":"2024-04-26T21:45:57.688536+0000","last_became_active":"2024-04-26T21:45:56.699515+0000","last_became_peered":"2024-04-26T21:45:56.699515+0000","last_unstale":"2024-04-26T21:45:57.688536+0000","last_undegraded":"2024-04-26T21:45:57.688536+0000","last_fullsized":"2024-04-26T21:45:57.688536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784161+0000","last_change":"2024-04-26T21:45:56.696080+0000","last_active":"2024-04-26T21:45:57.784161+0000","last_peered":"2024-04-26T21:45:57.784161+0000","last_clean":"2024-04-26T21:45:57.784161+0000","last_became_active":"2024-04-26T21:45:56.695767+0000","last_became_peered":"2024-04-26T21:45:56.695767+0000","last_unstale":"2024-04-26T21:45:57.784161+0000","last_undegraded":"2024-04-26T21:45:57.784161+0000","last_fullsized":"2024-04-26T21:45:57.784161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786723+0000","last_change":"2024-04-26T21:45:56.724164+0000","last_active":"2024-04-26T21:45:57.786723+0000","last_peered":"2024-04-26T21:45:57.786723+0000","last_clean":"2024-04-26T21:45:57.786723+0000","last_became_active":"2024-04-26T21:45:56.719445+0000","last_became_peered":"2024-04-26T21:45:56.719445+0000","last_unstale":"2024-04-26T21:45:57.786723+0000","last_undegraded":"2024-04-26T21:45:57.786723+0000","last_fullsized":"2024-04-26T21:45:57.786723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697154+0000","last_change":"2024-04-26T21:45:56.692330+0000","last_active":"2024-04-26T21:45:57.697154+0000","last_peered":"2024-04-26T21:45:57.697154+0000","last_clean":"2024-04-26T21:45:57.697154+0000","last_became_active":"2024-04-26T21:45:56.692002+0000","last_became_peered":"2024-04-26T21:45:56.692002+0000","last_unstale":"2024-04-26T21:45:57.697154+0000","last_undegraded":"2024-04-26T21:45:57.697154+0000","last_fullsized":"2024-04-26T21:45:57.697154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786129+0000","last_change":"2024-04-26T21:45:56.719030+0000","last_active":"2024-04-26T21:45:57.786129+0000","last_peered":"2024-04-26T21:45:57.786129+0000","last_clean":"2024-04-26T21:45:57.786129+0000","last_became_active":"2024-04-26T21:45:56.718334+0000","last_became_peered":"2024-04-26T21:45:56.718334+0000","last_unstale":"2024-04-26T21:45:57.786129+0000","last_undegraded":"2024-04-26T21:45:57.786129+0000","last_fullsized":"2024-04-26T21:45:57.786129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696797+0000","last_change":"2024-04-26T21:45:56.692710+0000","last_active":"2024-04-26T21:45:57.696797+0000","last_peered":"2024-04-26T21:45:57.696797+0000","last_clean":"2024-04-26T21:45:57.696797+0000","last_became_active":"2024-04-26T21:45:56.692374+0000","last_became_peered":"2024-04-26T21:45:56.692374+0000","last_unstale":"2024-04-26T21:45:57.696797+0000","last_undegraded":"2024-04-26T21:45:57.696797+0000","last_fullsized":"2024-04-26T21:45:57.696797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783131+0000","last_change":"2024-04-26T21:45:56.718800+0000","last_active":"2024-04-26T21:45:57.783131+0000","last_peered":"2024-04-26T21:45:57.783131+0000","last_clean":"2024-04-26T21:45:57.783131+0000","last_became_active":"2024-04-26T21:45:56.717337+0000","last_became_peered":"2024-04-26T21:45:56.717337+0000","last_unstale":"2024-04-26T21:45:57.783131+0000","last_undegraded":"2024-04-26T21:45:57.783131+0000","last_fullsized":"2024-04-26T21:45:57.783131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698512+0000","last_change":"2024-04-26T21:45:56.722478+0000","last_active":"2024-04-26T21:45:57.698512+0000","last_peered":"2024-04-26T21:45:57.698512+0000","last_clean":"2024-04-26T21:45:57.698512+0000","last_became_active":"2024-04-26T21:45:56.722182+0000","last_became_peered":"2024-04-26T21:45:56.722182+0000","last_unstale":"2024-04-26T21:45:57.698512+0000","last_undegraded":"2024-04-26T21:45:57.698512+0000","last_fullsized":"2024-04-26T21:45:57.698512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:22.652553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706561+0000","last_change":"2024-04-26T21:45:56.716505+0000","last_active":"2024-04-26T21:45:57.706561+0000","last_peered":"2024-04-26T21:45:57.706561+0000","last_clean":"2024-04-26T21:45:57.706561+0000","last_became_active":"2024-04-26T21:45:56.716200+0000","last_became_peered":"2024-04-26T21:45:56.716200+0000","last_unstale":"2024-04-26T21:45:57.706561+0000","last_undegraded":"2024-04-26T21:45:57.706561+0000","last_fullsized":"2024-04-26T21:45:57.706561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706119+0000","last_change":"2024-04-26T21:45:56.719125+0000","last_active":"2024-04-26T21:45:57.706119+0000","last_peered":"2024-04-26T21:45:57.706119+0000","last_clean":"2024-04-26T21:45:57.706119+0000","last_became_active":"2024-04-26T21:45:56.718753+0000","last_became_peered":"2024-04-26T21:45:56.718753+0000","last_unstale":"2024-04-26T21:45:57.706119+0000","last_undegraded":"2024-04-26T21:45:57.706119+0000","last_fullsized":"2024-04-26T21:45:57.706119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694414+0000","last_change":"2024-04-26T21:45:56.703498+0000","last_active":"2024-04-26T21:45:57.694414+0000","last_peered":"2024-04-26T21:45:57.694414+0000","last_clean":"2024-04-26T21:45:57.694414+0000","last_became_active":"2024-04-26T21:45:56.702682+0000","last_became_peered":"2024-04-26T21:45:56.702682+0000","last_unstale":"2024-04-26T21:45:57.694414+0000","last_undegraded":"2024-04-26T21:45:57.694414+0000","last_fullsized":"2024-04-26T21:45:57.694414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783918+0000","last_change":"2024-04-26T21:45:56.690293+0000","last_active":"2024-04-26T21:45:57.783918+0000","last_peered":"2024-04-26T21:45:57.783918+0000","last_clean":"2024-04-26T21:45:57.783918+0000","last_became_active":"2024-04-26T21:45:56.689962+0000","last_became_peered":"2024-04-26T21:45:56.689962+0000","last_unstale":"2024-04-26T21:45:57.783918+0000","last_undegraded":"2024-04-26T21:45:57.783918+0000","last_fullsized":"2024-04-26T21:45:57.783918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783643+0000","last_change":"2024-04-26T21:45:56.701473+0000","last_active":"2024-04-26T21:45:57.783643+0000","last_peered":"2024-04-26T21:45:57.783643+0000","last_clean":"2024-04-26T21:45:57.783643+0000","last_became_active":"2024-04-26T21:45:56.697858+0000","last_became_peered":"2024-04-26T21:45:56.697858+0000","last_unstale":"2024-04-26T21:45:57.783643+0000","last_undegraded":"2024-04-26T21:45:57.783643+0000","last_fullsized":"2024-04-26T21:45:57.783643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:13.204924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687846+0000","last_change":"2024-04-26T21:45:56.684598+0000","last_active":"2024-04-26T21:45:57.687846+0000","last_peered":"2024-04-26T21:45:57.687846+0000","last_clean":"2024-04-26T21:45:57.687846+0000","last_became_active":"2024-04-26T21:45:56.684065+0000","last_became_peered":"2024-04-26T21:45:56.684065+0000","last_unstale":"2024-04-26T21:45:57.687846+0000","last_undegraded":"2024-04-26T21:45:57.687846+0000","last_fullsized":"2024-04-26T21:45:57.687846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785651+0000","last_change":"2024-04-26T21:45:56.713842+0000","last_active":"2024-04-26T21:45:57.785651+0000","last_peered":"2024-04-26T21:45:57.785651+0000","last_clean":"2024-04-26T21:45:57.785651+0000","last_became_active":"2024-04-26T21:45:56.713554+0000","last_became_peered":"2024-04-26T21:45:56.713554+0000","last_unstale":"2024-04-26T21:45:57.785651+0000","last_undegraded":"2024-04-26T21:45:57.785651+0000","last_fullsized":"2024-04-26T21:45:57.785651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.787120+0000","last_change":"2024-04-26T21:45:56.717288+0000","last_active":"2024-04-26T21:45:57.787120+0000","last_peered":"2024-04-26T21:45:57.787120+0000","last_clean":"2024-04-26T21:45:57.787120+0000","last_became_active":"2024-04-26T21:45:56.716846+0000","last_became_peered":"2024-04-26T21:45:56.716846+0000","last_unstale":"2024-04-26T21:45:57.787120+0000","last_undegraded":"2024-04-26T21:45:57.787120+0000","last_fullsized":"2024-04-26T21:45:57.787120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782583+0000","last_change":"2024-04-26T21:45:56.709803+0000","last_active":"2024-04-26T21:45:57.782583+0000","last_peered":"2024-04-26T21:45:57.782583+0000","last_clean":"2024-04-26T21:45:57.782583+0000","last_became_active":"2024-04-26T21:45:56.709518+0000","last_became_peered":"2024-04-26T21:45:56.709518+0000","last_unstale":"2024-04-26T21:45:57.782583+0000","last_undegraded":"2024-04-26T21:45:57.782583+0000","last_fullsized":"2024-04-26T21:45:57.782583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697786+0000","last_change":"2024-04-26T21:45:56.691493+0000","last_active":"2024-04-26T21:45:57.697786+0000","last_peered":"2024-04-26T21:45:57.697786+0000","last_clean":"2024-04-26T21:45:57.697786+0000","last_became_active":"2024-04-26T21:45:56.691171+0000","last_became_peered":"2024-04-26T21:45:56.691171+0000","last_unstale":"2024-04-26T21:45:57.697786+0000","last_undegraded":"2024-04-26T21:45:57.697786+0000","last_fullsized":"2024-04-26T21:45:57.697786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706090+0000","last_change":"2024-04-26T21:45:56.696933+0000","last_active":"2024-04-26T21:45:57.706090+0000","last_peered":"2024-04-26T21:45:57.706090+0000","last_clean":"2024-04-26T21:45:57.706090+0000","last_became_active":"2024-04-26T21:45:56.696509+0000","last_became_peered":"2024-04-26T21:45:56.696509+0000","last_unstale":"2024-04-26T21:45:57.706090+0000","last_undegraded":"2024-04-26T21:45:57.706090+0000","last_fullsized":"2024-04-26T21:45:57.706090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784228+0000","last_change":"2024-04-26T21:45:56.696412+0000","last_active":"2024-04-26T21:45:57.784228+0000","last_peered":"2024-04-26T21:45:57.784228+0000","last_clean":"2024-04-26T21:45:57.784228+0000","last_became_active":"2024-04-26T21:45:56.695758+0000","last_became_peered":"2024-04-26T21:45:56.695758+0000","last_unstale":"2024-04-26T21:45:57.784228+0000","last_undegraded":"2024-04-26T21:45:57.784228+0000","last_fullsized":"2024-04-26T21:45:57.784228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786146+0000","last_change":"2024-04-26T21:45:56.717913+0000","last_active":"2024-04-26T21:45:57.786146+0000","last_peered":"2024-04-26T21:45:57.786146+0000","last_clean":"2024-04-26T21:45:57.786146+0000","last_became_active":"2024-04-26T21:45:56.717579+0000","last_became_peered":"2024-04-26T21:45:56.717579+0000","last_unstale":"2024-04-26T21:45:57.786146+0000","last_undegraded":"2024-04-26T21:45:57.786146+0000","last_fullsized":"2024-04-26T21:45:57.786146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783112+0000","last_change":"2024-04-26T21:45:56.694204+0000","last_active":"2024-04-26T21:45:57.783112+0000","last_peered":"2024-04-26T21:45:57.783112+0000","last_clean":"2024-04-26T21:45:57.783112+0000","last_became_active":"2024-04-26T21:45:56.693845+0000","last_became_peered":"2024-04-26T21:45:56.693845+0000","last_unstale":"2024-04-26T21:45:57.783112+0000","last_undegraded":"2024-04-26T21:45:57.783112+0000","last_fullsized":"2024-04-26T21:45:57.783112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785959+0000","last_change":"2024-04-26T21:45:56.710113+0000","last_active":"2024-04-26T21:45:57.785959+0000","last_peered":"2024-04-26T21:45:57.785959+0000","last_clean":"2024-04-26T21:45:57.785959+0000","last_became_active":"2024-04-26T21:45:56.709659+0000","last_became_peered":"2024-04-26T21:45:56.709659+0000","last_unstale":"2024-04-26T21:45:57.785959+0000","last_undegraded":"2024-04-26T21:45:57.785959+0000","last_fullsized":"2024-04-26T21:45:57.785959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784144+0000","last_change":"2024-04-26T21:45:56.693549+0000","last_active":"2024-04-26T21:45:57.784144+0000","last_peered":"2024-04-26T21:45:57.784144+0000","last_clean":"2024-04-26T21:45:57.784144+0000","last_became_active":"2024-04-26T21:45:56.693216+0000","last_became_peered":"2024-04-26T21:45:56.693216+0000","last_unstale":"2024-04-26T21:45:57.784144+0000","last_undegraded":"2024-04-26T21:45:57.784144+0000","last_fullsized":"2024-04-26T21:45:57.784144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706610+0000","last_change":"2024-04-26T21:45:56.697472+0000","last_active":"2024-04-26T21:45:57.706610+0000","last_peered":"2024-04-26T21:45:57.706610+0000","last_clean":"2024-04-26T21:45:57.706610+0000","last_became_active":"2024-04-26T21:45:56.697216+0000","last_became_peered":"2024-04-26T21:45:56.697216+0000","last_unstale":"2024-04-26T21:45:57.706610+0000","last_undegraded":"2024-04-26T21:45:57.706610+0000","last_fullsized":"2024-04-26T21:45:57.706610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688367+0000","last_change":"2024-04-26T21:45:56.696018+0000","last_active":"2024-04-26T21:45:57.688367+0000","last_peered":"2024-04-26T21:45:57.688367+0000","last_clean":"2024-04-26T21:45:57.688367+0000","last_became_active":"2024-04-26T21:45:56.695689+0000","last_became_peered":"2024-04-26T21:45:56.695689+0000","last_unstale":"2024-04-26T21:45:57.688367+0000","last_undegraded":"2024-04-26T21:45:57.688367+0000","last_fullsized":"2024-04-26T21:45:57.688367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784032+0000","last_change":"2024-04-26T21:45:56.694683+0000","last_active":"2024-04-26T21:45:57.784032+0000","last_peered":"2024-04-26T21:45:57.784032+0000","last_clean":"2024-04-26T21:45:57.784032+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.784032+0000","last_undegraded":"2024-04-26T21:45:57.784032+0000","last_fullsized":"2024-04-26T21:45:57.784032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784298+0000","last_change":"2024-04-26T21:45:56.708141+0000","last_active":"2024-04-26T21:45:57.784298+0000","last_peered":"2024-04-26T21:45:57.784298+0000","last_clean":"2024-04-26T21:45:57.784298+0000","last_became_active":"2024-04-26T21:45:56.705466+0000","last_became_peered":"2024-04-26T21:45:56.705466+0000","last_unstale":"2024-04-26T21:45:57.784298+0000","last_undegraded":"2024-04-26T21:45:57.784298+0000","last_fullsized":"2024-04-26T21:45:57.784298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786657+0000","last_change":"2024-04-26T21:45:56.719370+0000","last_active":"2024-04-26T21:45:57.786657+0000","last_peered":"2024-04-26T21:45:57.786657+0000","last_clean":"2024-04-26T21:45:57.786657+0000","last_became_active":"2024-04-26T21:45:56.718686+0000","last_became_peered":"2024-04-26T21:45:56.718686+0000","last_unstale":"2024-04-26T21:45:57.786657+0000","last_undegraded":"2024-04-26T21:45:57.786657+0000","last_fullsized":"2024-04-26T21:45:57.786657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782446+0000","last_change":"2024-04-26T21:45:56.716691+0000","last_active":"2024-04-26T21:45:57.782446+0000","last_peered":"2024-04-26T21:45:57.782446+0000","last_clean":"2024-04-26T21:45:57.782446+0000","last_became_active":"2024-04-26T21:45:56.716457+0000","last_became_peered":"2024-04-26T21:45:56.716457+0000","last_unstale":"2024-04-26T21:45:57.782446+0000","last_undegraded":"2024-04-26T21:45:57.782446+0000","last_fullsized":"2024-04-26T21:45:57.782446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706078+0000","last_change":"2024-04-26T21:45:56.693796+0000","last_active":"2024-04-26T21:45:57.706078+0000","last_peered":"2024-04-26T21:45:57.706078+0000","last_clean":"2024-04-26T21:45:57.706078+0000","last_became_active":"2024-04-26T21:45:56.693393+0000","last_became_peered":"2024-04-26T21:45:56.693393+0000","last_unstale":"2024-04-26T21:45:57.706078+0000","last_undegraded":"2024-04-26T21:45:57.706078+0000","last_fullsized":"2024-04-26T21:45:57.706078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693646+0000","last_change":"2024-04-26T21:45:56.684700+0000","last_active":"2024-04-26T21:45:57.693646+0000","last_peered":"2024-04-26T21:45:57.693646+0000","last_clean":"2024-04-26T21:45:57.693646+0000","last_became_active":"2024-04-26T21:45:56.683789+0000","last_became_peered":"2024-04-26T21:45:56.683789+0000","last_unstale":"2024-04-26T21:45:57.693646+0000","last_undegraded":"2024-04-26T21:45:57.693646+0000","last_fullsized":"2024-04-26T21:45:57.693646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:48.558636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783980+0000","last_change":"2024-04-26T21:45:55.672715+0000","last_active":"2024-04-26T21:45:57.783980+0000","last_peered":"2024-04-26T21:45:57.783980+0000","last_clean":"2024-04-26T21:45:57.783980+0000","last_became_active":"2024-04-26T21:45:55.672430+0000","last_became_peered":"2024-04-26T21:45:55.672430+0000","last_unstale":"2024-04-26T21:45:57.783980+0000","last_undegraded":"2024-04-26T21:45:57.783980+0000","last_fullsized":"2024-04-26T21:45:57.783980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784365+0000","last_change":"2024-04-26T21:45:56.697463+0000","last_active":"2024-04-26T21:45:57.784365+0000","last_peered":"2024-04-26T21:45:57.784365+0000","last_clean":"2024-04-26T21:45:57.784365+0000","last_became_active":"2024-04-26T21:45:56.697124+0000","last_became_peered":"2024-04-26T21:45:56.697124+0000","last_unstale":"2024-04-26T21:45:57.784365+0000","last_undegraded":"2024-04-26T21:45:57.784365+0000","last_fullsized":"2024-04-26T21:45:57.784365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705510+0000","last_change":"2024-04-26T21:45:55.678579+0000","last_active":"2024-04-26T21:45:57.705510+0000","last_peered":"2024-04-26T21:45:57.705510+0000","last_clean":"2024-04-26T21:45:57.705510+0000","last_became_active":"2024-04-26T21:45:55.678193+0000","last_became_peered":"2024-04-26T21:45:55.678193+0000","last_unstale":"2024-04-26T21:45:57.705510+0000","last_undegraded":"2024-04-26T21:45:57.705510+0000","last_fullsized":"2024-04-26T21:45:57.705510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783216+0000","last_change":"2024-04-26T21:45:55.675408+0000","last_active":"2024-04-26T21:45:57.783216+0000","last_peered":"2024-04-26T21:45:57.783216+0000","last_clean":"2024-04-26T21:45:57.783216+0000","last_became_active":"2024-04-26T21:45:55.675074+0000","last_became_peered":"2024-04-26T21:45:55.675074+0000","last_unstale":"2024-04-26T21:45:57.783216+0000","last_undegraded":"2024-04-26T21:45:57.783216+0000","last_fullsized":"2024-04-26T21:45:57.783216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783259+0000","last_change":"2024-04-26T21:45:56.694914+0000","last_active":"2024-04-26T21:45:57.783259+0000","last_peered":"2024-04-26T21:45:57.783259+0000","last_clean":"2024-04-26T21:45:57.783259+0000","last_became_active":"2024-04-26T21:45:56.694634+0000","last_became_peered":"2024-04-26T21:45:56.694634+0000","last_unstale":"2024-04-26T21:45:57.783259+0000","last_undegraded":"2024-04-26T21:45:57.783259+0000","last_fullsized":"2024-04-26T21:45:57.783259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785786+0000","last_change":"2024-04-26T21:45:55.669307+0000","last_active":"2024-04-26T21:45:57.785786+0000","last_peered":"2024-04-26T21:45:57.785786+0000","last_clean":"2024-04-26T21:45:57.785786+0000","last_became_active":"2024-04-26T21:45:55.668979+0000","last_became_peered":"2024-04-26T21:45:55.668979+0000","last_unstale":"2024-04-26T21:45:57.785786+0000","last_undegraded":"2024-04-26T21:45:57.785786+0000","last_fullsized":"2024-04-26T21:45:57.785786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687926+0000","last_change":"2024-04-26T21:45:56.697521+0000","last_active":"2024-04-26T21:45:57.687926+0000","last_peered":"2024-04-26T21:45:57.687926+0000","last_clean":"2024-04-26T21:45:57.687926+0000","last_became_active":"2024-04-26T21:45:56.697165+0000","last_became_peered":"2024-04-26T21:45:56.697165+0000","last_unstale":"2024-04-26T21:45:57.687926+0000","last_undegraded":"2024-04-26T21:45:57.687926+0000","last_fullsized":"2024-04-26T21:45:57.687926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706395+0000","last_change":"2024-04-26T21:45:55.686208+0000","last_active":"2024-04-26T21:45:57.706395+0000","last_peered":"2024-04-26T21:45:57.706395+0000","last_clean":"2024-04-26T21:45:57.706395+0000","last_became_active":"2024-04-26T21:45:55.685947+0000","last_became_peered":"2024-04-26T21:45:55.685947+0000","last_unstale":"2024-04-26T21:45:57.706395+0000","last_undegraded":"2024-04-26T21:45:57.706395+0000","last_fullsized":"2024-04-26T21:45:57.706395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786494+0000","last_change":"2024-04-26T21:45:56.714928+0000","last_active":"2024-04-26T21:45:57.786494+0000","last_peered":"2024-04-26T21:45:57.786494+0000","last_clean":"2024-04-26T21:45:57.786494+0000","last_became_active":"2024-04-26T21:45:56.714595+0000","last_became_peered":"2024-04-26T21:45:56.714595+0000","last_unstale":"2024-04-26T21:45:57.786494+0000","last_undegraded":"2024-04-26T21:45:57.786494+0000","last_fullsized":"2024-04-26T21:45:57.786494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786194+0000","last_change":"2024-04-26T21:45:55.684279+0000","last_active":"2024-04-26T21:45:57.786194+0000","last_peered":"2024-04-26T21:45:57.786194+0000","last_clean":"2024-04-26T21:45:57.786194+0000","last_became_active":"2024-04-26T21:45:55.683712+0000","last_became_peered":"2024-04-26T21:45:55.683712+0000","last_unstale":"2024-04-26T21:45:57.786194+0000","last_undegraded":"2024-04-26T21:45:57.786194+0000","last_fullsized":"2024-04-26T21:45:57.786194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784002+0000","last_change":"2024-04-26T21:45:56.719558+0000","last_active":"2024-04-26T21:45:57.784002+0000","last_peered":"2024-04-26T21:45:57.784002+0000","last_clean":"2024-04-26T21:45:57.784002+0000","last_became_active":"2024-04-26T21:45:56.718765+0000","last_became_peered":"2024-04-26T21:45:56.718765+0000","last_unstale":"2024-04-26T21:45:57.784002+0000","last_undegraded":"2024-04-26T21:45:57.784002+0000","last_fullsized":"2024-04-26T21:45:57.784002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688698+0000","last_change":"2024-04-26T21:45:55.682896+0000","last_active":"2024-04-26T21:45:57.688698+0000","last_peered":"2024-04-26T21:45:57.688698+0000","last_clean":"2024-04-26T21:45:57.688698+0000","last_became_active":"2024-04-26T21:45:55.682576+0000","last_became_peered":"2024-04-26T21:45:55.682576+0000","last_unstale":"2024-04-26T21:45:57.688698+0000","last_undegraded":"2024-04-26T21:45:57.688698+0000","last_fullsized":"2024-04-26T21:45:57.688698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694424+0000","last_change":"2024-04-26T21:45:56.702692+0000","last_active":"2024-04-26T21:45:57.694424+0000","last_peered":"2024-04-26T21:45:57.694424+0000","last_clean":"2024-04-26T21:45:57.694424+0000","last_became_active":"2024-04-26T21:45:56.702369+0000","last_became_peered":"2024-04-26T21:45:56.702369+0000","last_unstale":"2024-04-26T21:45:57.694424+0000","last_undegraded":"2024-04-26T21:45:57.694424+0000","last_fullsized":"2024-04-26T21:45:57.694424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705841+0000","last_change":"2024-04-26T21:45:55.678223+0000","last_active":"2024-04-26T21:45:57.705841+0000","last_peered":"2024-04-26T21:45:57.705841+0000","last_clean":"2024-04-26T21:45:57.705841+0000","last_became_active":"2024-04-26T21:45:55.676989+0000","last_became_peered":"2024-04-26T21:45:55.676989+0000","last_unstale":"2024-04-26T21:45:57.705841+0000","last_undegraded":"2024-04-26T21:45:57.705841+0000","last_fullsized":"2024-04-26T21:45:57.705841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697495+0000","last_change":"2024-04-26T21:45:56.700587+0000","last_active":"2024-04-26T21:45:57.697495+0000","last_peered":"2024-04-26T21:45:57.697495+0000","last_clean":"2024-04-26T21:45:57.697495+0000","last_became_active":"2024-04-26T21:45:56.695698+0000","last_became_peered":"2024-04-26T21:45:56.695698+0000","last_unstale":"2024-04-26T21:45:57.697495+0000","last_undegraded":"2024-04-26T21:45:57.697495+0000","last_fullsized":"2024-04-26T21:45:57.697495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784401+0000","last_change":"2024-04-26T21:45:55.674764+0000","last_active":"2024-04-26T21:45:57.784401+0000","last_peered":"2024-04-26T21:45:57.784401+0000","last_clean":"2024-04-26T21:45:57.784401+0000","last_became_active":"2024-04-26T21:45:55.674470+0000","last_became_peered":"2024-04-26T21:45:55.674470+0000","last_unstale":"2024-04-26T21:45:57.784401+0000","last_undegraded":"2024-04-26T21:45:57.784401+0000","last_fullsized":"2024-04-26T21:45:57.784401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783659+0000","last_change":"2024-04-26T21:45:56.702112+0000","last_active":"2024-04-26T21:45:57.783659+0000","last_peered":"2024-04-26T21:45:57.783659+0000","last_clean":"2024-04-26T21:45:57.783659+0000","last_became_active":"2024-04-26T21:45:56.701847+0000","last_became_peered":"2024-04-26T21:45:56.701847+0000","last_unstale":"2024-04-26T21:45:57.783659+0000","last_undegraded":"2024-04-26T21:45:57.783659+0000","last_fullsized":"2024-04-26T21:45:57.783659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786937+0000","last_change":"2024-04-26T21:45:55.674485+0000","last_active":"2024-04-26T21:45:57.786937+0000","last_peered":"2024-04-26T21:45:57.786937+0000","last_clean":"2024-04-26T21:45:57.786937+0000","last_became_active":"2024-04-26T21:45:55.674236+0000","last_became_peered":"2024-04-26T21:45:55.674236+0000","last_unstale":"2024-04-26T21:45:57.786937+0000","last_undegraded":"2024-04-26T21:45:57.786937+0000","last_fullsized":"2024-04-26T21:45:57.786937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697306+0000","last_change":"2024-04-26T21:45:56.691631+0000","last_active":"2024-04-26T21:45:57.697306+0000","last_peered":"2024-04-26T21:45:57.697306+0000","last_clean":"2024-04-26T21:45:57.697306+0000","last_became_active":"2024-04-26T21:45:56.691262+0000","last_became_peered":"2024-04-26T21:45:56.691262+0000","last_unstale":"2024-04-26T21:45:57.697306+0000","last_undegraded":"2024-04-26T21:45:57.697306+0000","last_fullsized":"2024-04-26T21:45:57.697306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705443+0000","last_change":"2024-04-26T21:45:55.679236+0000","last_active":"2024-04-26T21:45:57.705443+0000","last_peered":"2024-04-26T21:45:57.705443+0000","last_clean":"2024-04-26T21:45:57.705443+0000","last_became_active":"2024-04-26T21:45:55.678916+0000","last_became_peered":"2024-04-26T21:45:55.678916+0000","last_unstale":"2024-04-26T21:45:57.705443+0000","last_undegraded":"2024-04-26T21:45:57.705443+0000","last_fullsized":"2024-04-26T21:45:57.705443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705838+0000","last_change":"2024-04-26T21:45:56.693186+0000","last_active":"2024-04-26T21:45:57.705838+0000","last_peered":"2024-04-26T21:45:57.705838+0000","last_clean":"2024-04-26T21:45:57.705838+0000","last_became_active":"2024-04-26T21:45:56.692617+0000","last_became_peered":"2024-04-26T21:45:56.692617+0000","last_unstale":"2024-04-26T21:45:57.705838+0000","last_undegraded":"2024-04-26T21:45:57.705838+0000","last_fullsized":"2024-04-26T21:45:57.705838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782566+0000","last_change":"2024-04-26T21:45:55.669857+0000","last_active":"2024-04-26T21:45:57.782566+0000","last_peered":"2024-04-26T21:45:57.782566+0000","last_clean":"2024-04-26T21:45:57.782566+0000","last_became_active":"2024-04-26T21:45:55.667192+0000","last_became_peered":"2024-04-26T21:45:55.667192+0000","last_unstale":"2024-04-26T21:45:57.782566+0000","last_undegraded":"2024-04-26T21:45:57.782566+0000","last_fullsized":"2024-04-26T21:45:57.782566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696941+0000","last_change":"2024-04-26T21:45:56.688026+0000","last_active":"2024-04-26T21:45:57.696941+0000","last_peered":"2024-04-26T21:45:57.696941+0000","last_clean":"2024-04-26T21:45:57.696941+0000","last_became_active":"2024-04-26T21:45:56.687620+0000","last_became_peered":"2024-04-26T21:45:56.687620+0000","last_unstale":"2024-04-26T21:45:57.696941+0000","last_undegraded":"2024-04-26T21:45:57.696941+0000","last_fullsized":"2024-04-26T21:45:57.696941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694496+0000","last_change":"2024-04-26T21:45:55.671925+0000","last_active":"2024-04-26T21:45:57.694496+0000","last_peered":"2024-04-26T21:45:57.694496+0000","last_clean":"2024-04-26T21:45:57.694496+0000","last_became_active":"2024-04-26T21:45:55.671562+0000","last_became_peered":"2024-04-26T21:45:55.671562+0000","last_unstale":"2024-04-26T21:45:57.694496+0000","last_undegraded":"2024-04-26T21:45:57.694496+0000","last_fullsized":"2024-04-26T21:45:57.694496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:47.590826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697702+0000","last_change":"2024-04-26T21:45:56.696591+0000","last_active":"2024-04-26T21:45:57.697702+0000","last_peered":"2024-04-26T21:45:57.697702+0000","last_clean":"2024-04-26T21:45:57.697702+0000","last_became_active":"2024-04-26T21:45:56.695741+0000","last_became_peered":"2024-04-26T21:45:56.695741+0000","last_unstale":"2024-04-26T21:45:57.697702+0000","last_undegraded":"2024-04-26T21:45:57.697702+0000","last_fullsized":"2024-04-26T21:45:57.697702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782611+0000","last_change":"2024-04-26T21:45:55.661721+0000","last_active":"2024-04-26T21:45:57.782611+0000","last_peered":"2024-04-26T21:45:57.782611+0000","last_clean":"2024-04-26T21:45:57.782611+0000","last_became_active":"2024-04-26T21:45:55.661317+0000","last_became_peered":"2024-04-26T21:45:55.661317+0000","last_unstale":"2024-04-26T21:45:57.782611+0000","last_undegraded":"2024-04-26T21:45:57.782611+0000","last_fullsized":"2024-04-26T21:45:57.782611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698417+0000","last_change":"2024-04-26T21:45:56.690727+0000","last_active":"2024-04-26T21:45:57.698417+0000","last_peered":"2024-04-26T21:45:57.698417+0000","last_clean":"2024-04-26T21:45:57.698417+0000","last_became_active":"2024-04-26T21:45:56.689340+0000","last_became_peered":"2024-04-26T21:45:56.689340+0000","last_unstale":"2024-04-26T21:45:57.698417+0000","last_undegraded":"2024-04-26T21:45:57.698417+0000","last_fullsized":"2024-04-26T21:45:57.698417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:16.382523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688598+0000","last_change":"2024-04-26T21:45:55.667840+0000","last_active":"2024-04-26T21:45:57.688598+0000","last_peered":"2024-04-26T21:45:57.688598+0000","last_clean":"2024-04-26T21:45:57.688598+0000","last_became_active":"2024-04-26T21:45:55.667095+0000","last_became_peered":"2024-04-26T21:45:55.667095+0000","last_unstale":"2024-04-26T21:45:57.688598+0000","last_undegraded":"2024-04-26T21:45:57.688598+0000","last_fullsized":"2024-04-26T21:45:57.688598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786997+0000","last_change":"2024-04-26T21:45:56.724926+0000","last_active":"2024-04-26T21:45:57.786997+0000","last_peered":"2024-04-26T21:45:57.786997+0000","last_clean":"2024-04-26T21:45:57.786997+0000","last_became_active":"2024-04-26T21:45:56.723585+0000","last_became_peered":"2024-04-26T21:45:56.723585+0000","last_unstale":"2024-04-26T21:45:57.786997+0000","last_undegraded":"2024-04-26T21:45:57.786997+0000","last_fullsized":"2024-04-26T21:45:57.786997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782488+0000","last_change":"2024-04-26T21:45:55.679971+0000","last_active":"2024-04-26T21:45:57.782488+0000","last_peered":"2024-04-26T21:45:57.782488+0000","last_clean":"2024-04-26T21:45:57.782488+0000","last_became_active":"2024-04-26T21:45:55.679650+0000","last_became_peered":"2024-04-26T21:45:55.679650+0000","last_unstale":"2024-04-26T21:45:57.782488+0000","last_undegraded":"2024-04-26T21:45:57.782488+0000","last_fullsized":"2024-04-26T21:45:57.782488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697335+0000","last_change":"2024-04-26T21:45:56.707369+0000","last_active":"2024-04-26T21:45:57.697335+0000","last_peered":"2024-04-26T21:45:57.697335+0000","last_clean":"2024-04-26T21:45:57.697335+0000","last_became_active":"2024-04-26T21:45:56.707148+0000","last_became_peered":"2024-04-26T21:45:56.707148+0000","last_unstale":"2024-04-26T21:45:57.697335+0000","last_undegraded":"2024-04-26T21:45:57.697335+0000","last_fullsized":"2024-04-26T21:45:57.697335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696696+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:45:57.696696+0000","last_peered":"2024-04-26T21:45:57.696696+0000","last_clean":"2024-04-26T21:45:57.696696+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:45:57.696696+0000","last_undegraded":"2024-04-26T21:45:57.696696+0000","last_fullsized":"2024-04-26T21:45:57.696696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785566+0000","last_change":"2024-04-26T21:45:55.671235+0000","last_active":"2024-04-26T21:45:57.785566+0000","last_peered":"2024-04-26T21:45:57.785566+0000","last_clean":"2024-04-26T21:45:57.785566+0000","last_became_active":"2024-04-26T21:45:55.670854+0000","last_became_peered":"2024-04-26T21:45:55.670854+0000","last_unstale":"2024-04-26T21:45:57.785566+0000","last_undegraded":"2024-04-26T21:45:57.785566+0000","last_fullsized":"2024-04-26T21:45:57.785566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783861+0000","last_change":"2024-04-26T21:45:56.723841+0000","last_active":"2024-04-26T21:45:57.783861+0000","last_peered":"2024-04-26T21:45:57.783861+0000","last_clean":"2024-04-26T21:45:57.783861+0000","last_became_active":"2024-04-26T21:45:56.723301+0000","last_became_peered":"2024-04-26T21:45:56.723301+0000","last_unstale":"2024-04-26T21:45:57.783861+0000","last_undegraded":"2024-04-26T21:45:57.783861+0000","last_fullsized":"2024-04-26T21:45:57.783861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696426+0000","last_change":"2024-04-26T21:45:55.669460+0000","last_active":"2024-04-26T21:45:57.696426+0000","last_peered":"2024-04-26T21:45:57.696426+0000","last_clean":"2024-04-26T21:45:57.696426+0000","last_became_active":"2024-04-26T21:45:55.668390+0000","last_became_peered":"2024-04-26T21:45:55.668390+0000","last_unstale":"2024-04-26T21:45:57.696426+0000","last_undegraded":"2024-04-26T21:45:57.696426+0000","last_fullsized":"2024-04-26T21:45:57.696426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783571+0000","last_change":"2024-04-26T21:45:56.700088+0000","last_active":"2024-04-26T21:45:57.783571+0000","last_peered":"2024-04-26T21:45:57.783571+0000","last_clean":"2024-04-26T21:45:57.783571+0000","last_became_active":"2024-04-26T21:45:56.699757+0000","last_became_peered":"2024-04-26T21:45:56.699757+0000","last_unstale":"2024-04-26T21:45:57.783571+0000","last_undegraded":"2024-04-26T21:45:57.783571+0000","last_fullsized":"2024-04-26T21:45:57.783571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:57.821808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782410+0000","last_change":"2024-04-26T21:45:55.662154+0000","last_active":"2024-04-26T21:45:57.782410+0000","last_peered":"2024-04-26T21:45:57.782410+0000","last_clean":"2024-04-26T21:45:57.782410+0000","last_became_active":"2024-04-26T21:45:55.661829+0000","last_became_peered":"2024-04-26T21:45:55.661829+0000","last_unstale":"2024-04-26T21:45:57.782410+0000","last_undegraded":"2024-04-26T21:45:57.782410+0000","last_fullsized":"2024-04-26T21:45:57.782410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688590+0000","last_change":"2024-04-26T21:45:56.689249+0000","last_active":"2024-04-26T21:45:57.688590+0000","last_peered":"2024-04-26T21:45:57.688590+0000","last_clean":"2024-04-26T21:45:57.688590+0000","last_became_active":"2024-04-26T21:45:56.688918+0000","last_became_peered":"2024-04-26T21:45:56.688918+0000","last_unstale":"2024-04-26T21:45:57.688590+0000","last_undegraded":"2024-04-26T21:45:57.688590+0000","last_fullsized":"2024-04-26T21:45:57.688590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786809+0000","last_change":"2024-04-26T21:45:55.672652+0000","last_active":"2024-04-26T21:45:57.786809+0000","last_peered":"2024-04-26T21:45:57.786809+0000","last_clean":"2024-04-26T21:45:57.786809+0000","last_became_active":"2024-04-26T21:45:55.672091+0000","last_became_peered":"2024-04-26T21:45:55.672091+0000","last_unstale":"2024-04-26T21:45:57.786809+0000","last_undegraded":"2024-04-26T21:45:57.786809+0000","last_fullsized":"2024-04-26T21:45:57.786809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:12.199574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693856+0000","last_change":"2024-04-26T21:45:56.697429+0000","last_active":"2024-04-26T21:45:57.693856+0000","last_peered":"2024-04-26T21:45:57.693856+0000","last_clean":"2024-04-26T21:45:57.693856+0000","last_became_active":"2024-04-26T21:45:56.697080+0000","last_became_peered":"2024-04-26T21:45:56.697080+0000","last_unstale":"2024-04-26T21:45:57.693856+0000","last_undegraded":"2024-04-26T21:45:57.693856+0000","last_fullsized":"2024-04-26T21:45:57.693856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:50.881602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687872+0000","last_change":"2024-04-26T21:45:55.667311+0000","last_active":"2024-04-26T21:45:57.687872+0000","last_peered":"2024-04-26T21:45:57.687872+0000","last_clean":"2024-04-26T21:45:57.687872+0000","last_became_active":"2024-04-26T21:45:55.667025+0000","last_became_peered":"2024-04-26T21:45:55.667025+0000","last_unstale":"2024-04-26T21:45:57.687872+0000","last_undegraded":"2024-04-26T21:45:57.687872+0000","last_fullsized":"2024-04-26T21:45:57.687872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786703+0000","last_change":"2024-04-26T21:45:56.714921+0000","last_active":"2024-04-26T21:45:57.786703+0000","last_peered":"2024-04-26T21:45:57.786703+0000","last_clean":"2024-04-26T21:45:57.786703+0000","last_became_active":"2024-04-26T21:45:56.714330+0000","last_became_peered":"2024-04-26T21:45:56.714330+0000","last_unstale":"2024-04-26T21:45:57.786703+0000","last_undegraded":"2024-04-26T21:45:57.786703+0000","last_fullsized":"2024-04-26T21:45:57.786703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697756+0000","last_change":"2024-04-26T21:45:55.673285+0000","last_active":"2024-04-26T21:45:57.697756+0000","last_peered":"2024-04-26T21:45:57.697756+0000","last_clean":"2024-04-26T21:45:57.697756+0000","last_became_active":"2024-04-26T21:45:55.672987+0000","last_became_peered":"2024-04-26T21:45:55.672987+0000","last_unstale":"2024-04-26T21:45:57.697756+0000","last_undegraded":"2024-04-26T21:45:57.697756+0000","last_fullsized":"2024-04-26T21:45:57.697756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:30.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687819+0000","last_change":"2024-04-26T21:45:56.696370+0000","last_active":"2024-04-26T21:45:57.687819+0000","last_peered":"2024-04-26T21:45:57.687819+0000","last_clean":"2024-04-26T21:45:57.687819+0000","last_became_active":"2024-04-26T21:45:56.696073+0000","last_became_peered":"2024-04-26T21:45:56.696073+0000","last_unstale":"2024-04-26T21:45:57.687819+0000","last_undegraded":"2024-04-26T21:45:57.687819+0000","last_fullsized":"2024-04-26T21:45:57.687819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688658+0000","last_change":"2024-04-26T21:45:55.683832+0000","last_active":"2024-04-26T21:45:57.688658+0000","last_peered":"2024-04-26T21:45:57.688658+0000","last_clean":"2024-04-26T21:45:57.688658+0000","last_became_active":"2024-04-26T21:45:55.683577+0000","last_became_peered":"2024-04-26T21:45:55.683577+0000","last_unstale":"2024-04-26T21:45:57.688658+0000","last_undegraded":"2024-04-26T21:45:57.688658+0000","last_fullsized":"2024-04-26T21:45:57.688658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784223+0000","last_change":"2024-04-26T21:45:56.695282+0000","last_active":"2024-04-26T21:45:57.784223+0000","last_peered":"2024-04-26T21:45:57.784223+0000","last_clean":"2024-04-26T21:45:57.784223+0000","last_became_active":"2024-04-26T21:45:56.694899+0000","last_became_peered":"2024-04-26T21:45:56.694899+0000","last_unstale":"2024-04-26T21:45:57.784223+0000","last_undegraded":"2024-04-26T21:45:57.784223+0000","last_fullsized":"2024-04-26T21:45:57.784223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706431+0000","last_change":"2024-04-26T21:45:55.684115+0000","last_active":"2024-04-26T21:45:57.706431+0000","last_peered":"2024-04-26T21:45:57.706431+0000","last_clean":"2024-04-26T21:45:57.706431+0000","last_became_active":"2024-04-26T21:45:55.683742+0000","last_became_peered":"2024-04-26T21:45:55.683742+0000","last_unstale":"2024-04-26T21:45:57.706431+0000","last_undegraded":"2024-04-26T21:45:57.706431+0000","last_fullsized":"2024-04-26T21:45:57.706431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783867+0000","last_change":"2024-04-26T21:45:56.710633+0000","last_active":"2024-04-26T21:45:57.783867+0000","last_peered":"2024-04-26T21:45:57.783867+0000","last_clean":"2024-04-26T21:45:57.783867+0000","last_became_active":"2024-04-26T21:45:56.710300+0000","last_became_peered":"2024-04-26T21:45:56.710300+0000","last_unstale":"2024-04-26T21:45:57.783867+0000","last_undegraded":"2024-04-26T21:45:57.783867+0000","last_fullsized":"2024-04-26T21:45:57.783867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696388+0000","last_change":"2024-04-26T21:45:55.669182+0000","last_active":"2024-04-26T21:45:57.696388+0000","last_peered":"2024-04-26T21:45:57.696388+0000","last_clean":"2024-04-26T21:45:57.696388+0000","last_became_active":"2024-04-26T21:45:55.668065+0000","last_became_peered":"2024-04-26T21:45:55.668065+0000","last_unstale":"2024-04-26T21:45:57.696388+0000","last_undegraded":"2024-04-26T21:45:57.696388+0000","last_fullsized":"2024-04-26T21:45:57.696388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693706+0000","last_change":"2024-04-26T21:45:56.690785+0000","last_active":"2024-04-26T21:45:57.693706+0000","last_peered":"2024-04-26T21:45:57.693706+0000","last_clean":"2024-04-26T21:45:57.693706+0000","last_became_active":"2024-04-26T21:45:56.690430+0000","last_became_peered":"2024-04-26T21:45:56.690430+0000","last_unstale":"2024-04-26T21:45:57.693706+0000","last_undegraded":"2024-04-26T21:45:57.693706+0000","last_fullsized":"2024-04-26T21:45:57.693706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:04.561079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687976+0000","last_change":"2024-04-26T21:45:55.668369+0000","last_active":"2024-04-26T21:45:57.687976+0000","last_peered":"2024-04-26T21:45:57.687976+0000","last_clean":"2024-04-26T21:45:57.687976+0000","last_became_active":"2024-04-26T21:45:55.667944+0000","last_became_peered":"2024-04-26T21:45:55.667944+0000","last_unstale":"2024-04-26T21:45:57.687976+0000","last_undegraded":"2024-04-26T21:45:57.687976+0000","last_fullsized":"2024-04-26T21:45:57.687976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783372+0000","last_change":"2024-04-26T21:45:56.703067+0000","last_active":"2024-04-26T21:45:57.783372+0000","last_peered":"2024-04-26T21:45:57.783372+0000","last_clean":"2024-04-26T21:45:57.783372+0000","last_became_active":"2024-04-26T21:45:56.702744+0000","last_became_peered":"2024-04-26T21:45:56.702744+0000","last_unstale":"2024-04-26T21:45:57.783372+0000","last_undegraded":"2024-04-26T21:45:57.783372+0000","last_fullsized":"2024-04-26T21:45:57.783372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688080+0000","last_change":"2024-04-26T21:45:55.669865+0000","last_active":"2024-04-26T21:45:57.688080+0000","last_peered":"2024-04-26T21:45:57.688080+0000","last_clean":"2024-04-26T21:45:57.688080+0000","last_became_active":"2024-04-26T21:45:55.669595+0000","last_became_peered":"2024-04-26T21:45:55.669595+0000","last_unstale":"2024-04-26T21:45:57.688080+0000","last_undegraded":"2024-04-26T21:45:57.688080+0000","last_fullsized":"2024-04-26T21:45:57.688080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783922+0000","last_change":"2024-04-26T21:45:56.699968+0000","last_active":"2024-04-26T21:45:57.783922+0000","last_peered":"2024-04-26T21:45:57.783922+0000","last_clean":"2024-04-26T21:45:57.783922+0000","last_became_active":"2024-04-26T21:45:56.699639+0000","last_became_peered":"2024-04-26T21:45:56.699639+0000","last_unstale":"2024-04-26T21:45:57.783922+0000","last_undegraded":"2024-04-26T21:45:57.783922+0000","last_fullsized":"2024-04-26T21:45:57.783922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697360+0000","last_change":"2024-04-26T21:45:55.673651+0000","last_active":"2024-04-26T21:45:57.697360+0000","last_peered":"2024-04-26T21:45:57.697360+0000","last_clean":"2024-04-26T21:45:57.697360+0000","last_became_active":"2024-04-26T21:45:55.673284+0000","last_became_peered":"2024-04-26T21:45:55.673284+0000","last_unstale":"2024-04-26T21:45:57.697360+0000","last_undegraded":"2024-04-26T21:45:57.697360+0000","last_fullsized":"2024-04-26T21:45:57.697360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705814+0000","last_change":"2024-04-26T21:45:56.694260+0000","last_active":"2024-04-26T21:45:57.705814+0000","last_peered":"2024-04-26T21:45:57.705814+0000","last_clean":"2024-04-26T21:45:57.705814+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.705814+0000","last_undegraded":"2024-04-26T21:45:57.705814+0000","last_fullsized":"2024-04-26T21:45:57.705814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786302+0000","last_change":"2024-04-26T21:45:56.715202+0000","last_active":"2024-04-26T21:45:57.786302+0000","last_peered":"2024-04-26T21:45:57.786302+0000","last_clean":"2024-04-26T21:45:57.786302+0000","last_became_active":"2024-04-26T21:45:56.714700+0000","last_became_peered":"2024-04-26T21:45:56.714700+0000","last_unstale":"2024-04-26T21:45:57.786302+0000","last_undegraded":"2024-04-26T21:45:57.786302+0000","last_fullsized":"2024-04-26T21:45:57.786302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697171+0000","last_change":"2024-04-26T21:45:55.681370+0000","last_active":"2024-04-26T21:45:57.697171+0000","last_peered":"2024-04-26T21:45:57.697171+0000","last_clean":"2024-04-26T21:45:57.697171+0000","last_became_active":"2024-04-26T21:45:55.681034+0000","last_became_peered":"2024-04-26T21:45:55.681034+0000","last_unstale":"2024-04-26T21:45:57.697171+0000","last_undegraded":"2024-04-26T21:45:57.697171+0000","last_fullsized":"2024-04-26T21:45:57.697171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786801+0000","last_change":"2024-04-26T21:45:56.713692+0000","last_active":"2024-04-26T21:45:57.786801+0000","last_peered":"2024-04-26T21:45:57.786801+0000","last_clean":"2024-04-26T21:45:57.786801+0000","last_became_active":"2024-04-26T21:45:56.713374+0000","last_became_peered":"2024-04-26T21:45:56.713374+0000","last_unstale":"2024-04-26T21:45:57.786801+0000","last_undegraded":"2024-04-26T21:45:57.786801+0000","last_fullsized":"2024-04-26T21:45:57.786801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697482+0000","last_change":"2024-04-26T21:45:55.679436+0000","last_active":"2024-04-26T21:45:57.697482+0000","last_peered":"2024-04-26T21:45:57.697482+0000","last_clean":"2024-04-26T21:45:57.697482+0000","last_became_active":"2024-04-26T21:45:55.678851+0000","last_became_peered":"2024-04-26T21:45:55.678851+0000","last_unstale":"2024-04-26T21:45:57.697482+0000","last_undegraded":"2024-04-26T21:45:57.697482+0000","last_fullsized":"2024-04-26T21:45:57.697482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:16.328469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783868+0000","last_change":"2024-04-26T21:45:56.719859+0000","last_active":"2024-04-26T21:45:57.783868+0000","last_peered":"2024-04-26T21:45:57.783868+0000","last_clean":"2024-04-26T21:45:57.783868+0000","last_became_active":"2024-04-26T21:45:56.719166+0000","last_became_peered":"2024-04-26T21:45:56.719166+0000","last_unstale":"2024-04-26T21:45:57.783868+0000","last_undegraded":"2024-04-26T21:45:57.783868+0000","last_fullsized":"2024-04-26T21:45:57.783868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784767+0000","last_change":"2024-04-26T21:45:55.664913+0000","last_active":"2024-04-26T21:45:57.784767+0000","last_peered":"2024-04-26T21:45:57.784767+0000","last_clean":"2024-04-26T21:45:57.784767+0000","last_became_active":"2024-04-26T21:45:55.664561+0000","last_became_peered":"2024-04-26T21:45:55.664561+0000","last_unstale":"2024-04-26T21:45:57.784767+0000","last_undegraded":"2024-04-26T21:45:57.784767+0000","last_fullsized":"2024-04-26T21:45:57.784767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784594+0000","last_change":"2024-04-26T21:45:56.698358+0000","last_active":"2024-04-26T21:45:57.784594+0000","last_peered":"2024-04-26T21:45:57.784594+0000","last_clean":"2024-04-26T21:45:57.784594+0000","last_became_active":"2024-04-26T21:45:56.697924+0000","last_became_peered":"2024-04-26T21:45:56.697924+0000","last_unstale":"2024-04-26T21:45:57.784594+0000","last_undegraded":"2024-04-26T21:45:57.784594+0000","last_fullsized":"2024-04-26T21:45:57.784594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783992+0000","last_change":"2024-04-26T21:45:55.668737+0000","last_active":"2024-04-26T21:45:57.783992+0000","last_peered":"2024-04-26T21:45:57.783992+0000","last_clean":"2024-04-26T21:45:57.783992+0000","last_became_active":"2024-04-26T21:45:55.668415+0000","last_became_peered":"2024-04-26T21:45:55.668415+0000","last_unstale":"2024-04-26T21:45:57.783992+0000","last_undegraded":"2024-04-26T21:45:57.783992+0000","last_fullsized":"2024-04-26T21:45:57.783992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:46:01.582 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:46:01.582 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:46:01.582 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:46:01.582 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T21:46:01.583 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T21:46:01.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.819+0000 7f461be00640 1 Processor -- start 2024-04-26T21:46:01.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.820+0000 7f461be00640 1 -- start start 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f461be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415f340 0x7f46140b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f461be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46140b5cf0 0x7f46140ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f461be00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140b4100 con 0x7f461415ef60 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f461be00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f46140b13d0 con 0x7f46140b5cf0 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f461be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46140b39e0 con 0x7f461415f340 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f4619400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f461415ef60 0x7f46140b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58792/0 (socket says 172.21.15.106:58792) 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f4619400640 1 -- 172.21.15.106:0/3087225777 learned_addr learned my addr 172.21.15.106:0/3087225777 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:01.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.821+0000 7f4619e00640 1 --2- 172.21.15.106:0/3087225777 >> v2:172.21.15.106:3300/0 conn(0x7f46140b5cf0 0x7f46140ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4613e00640 1 -- 172.21.15.106:0/3087225777 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46451924 0 0) 0x7f46140b4100 con 0x7f461415ef60 2024-04-26T21:46:01.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4613e00640 1 -- 172.21.15.106:0/3087225777 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45f8003520 con 0x7f461415ef60 2024-04-26T21:46:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4619e00640 1 -- 172.21.15.106:0/3087225777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415f340 msgr2=0x7f46140b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4619e00640 1 --2- 172.21.15.106:0/3087225777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415f340 0x7f46140b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4619e00640 1 -- 172.21.15.106:0/3087225777 >> v1:172.21.15.106:6789/0 conn(0x7f461415ef60 legacy=0x7f46140b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.822+0000 7f4619e00640 1 -- 172.21.15.106:0/3087225777 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140b4370 con 0x7f46140b5cf0 2024-04-26T21:46:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f4619e00640 1 --2- 172.21.15.106:0/3087225777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f46140ae040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c009d90 tx=0x7f460c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1684fe5a90c4c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f4613e00640 1 -- 172.21.15.106:0/3087225777 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c007740 con 0x7f46140b5cf0 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f4613e00640 1 -- 172.21.15.106:0/3087225777 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c0078e0 con 0x7f46140b5cf0 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f4613e00640 1 -- 172.21.15.106:0/3087225777 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c007cb0 con 0x7f46140b5cf0 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f461be00640 1 -- 172.21.15.106:0/3087225777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 msgr2=0x7f46140ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.823+0000 7f461be00640 1 --2- 172.21.15.106:0/3087225777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f46140ae040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460c009d90 tx=0x7f460c02a660 comp rx=0 tx=0).stop 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 -- 172.21.15.106:0/3087225777 shutdown_connections 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 --2- 172.21.15.106:0/3087225777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f46140ae040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 --2- 172.21.15.106:0/3087225777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415f340 0x7f46140b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 -- 172.21.15.106:0/3087225777 >> 172.21.15.106:0/3087225777 conn(0x7f46140b6580 msgr2=0x7f46140b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 -- 172.21.15.106:0/3087225777 shutdown_connections 2024-04-26T21:46:01.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.824+0000 7f461be00640 1 -- 172.21.15.106:0/3087225777 wait complete. 2024-04-26T21:46:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.825+0000 7f461be00640 1 Processor -- start 2024-04-26T21:46:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.825+0000 7f461be00640 1 -- start start 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415ef60 0x7f461415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461415f340 0x7f461414d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b1f70 con 0x7f46140b5cf0 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b62b0 con 0x7f461415f340 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.826+0000 7f461be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46140ae580 con 0x7f461415ef60 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51854/0 (socket says 172.21.15.106:51854) 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 -- 172.21.15.106:0/1400332933 learned_addr learned my addr 172.21.15.106:0/1400332933 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 -- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415ef60 msgr2=0x7f461415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415ef60 0x7f461415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 -- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461415f340 msgr2=0x7f461414d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461415f340 0x7f461414d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 -- 172.21.15.106:0/1400332933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461409bee0 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.827+0000 7f4619400640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f460400c430 tx=0x7f460400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.828+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604017070 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.828+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604005ce0 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.828+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040072f0 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.828+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461409e300 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.828+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461409bb60 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.829+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4614069540 con 0x7f46140b5cf0 2024-04-26T21:46:01.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.830+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f460401a070 con 0x7f46140b5cf0 2024-04-26T21:46:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.831+0000 7f4611600640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 0x7f45f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.831+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f460408af50 con 0x7f46140b5cf0 2024-04-26T21:46:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.831+0000 7f4618a00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 0x7f45f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.832+0000 7f4618a00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 0x7f45f406ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4614157720 tx=0x7f4600006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:01.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:01.835+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4604059410 con 0x7f46140b5cf0 2024-04-26T21:46:02.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.088+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f461406d0c0 con 0x7f46140b5cf0 2024-04-26T21:46:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.098+0000 7f4611600640 1 -- 172.21.15.106:0/1400332933 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7f4604058d10 con 0x7f46140b5cf0 2024-04-26T21:46:02.100 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 msgr2=0x7f45f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 0x7f45f406ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4614157720 tx=0x7f4600006bf0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 msgr2=0x7f461409ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f460400c430 tx=0x7f460400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 shutdown_connections 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461415f340 0x7f461414d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f45f406c920 0x7f45f406ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415ef60 0x7f461415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 --2- 172.21.15.106:0/1400332933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46140b5cf0 0x7f461409ede0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.102+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 >> 172.21.15.106:0/1400332933 conn(0x7f46140b6580 msgr2=0x7f461406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.103+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 shutdown_connections 2024-04-26T21:46:02.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.103+0000 7f461be00640 1 -- 172.21.15.106:0/1400332933 wait complete. 2024-04-26T21:46:02.121 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4027448399},{"type":"v1","addr":"172.21.15.106:6833","nonce":4027448399}]},"active_addr":"172.21.15.106:6833/4027448399","active_change":"2024-04-26T21:45:37.069518+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1345404225}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3323388612}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2266856270}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":455992440}]}]} 2024-04-26T21:46:02.125 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T21:46:02.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T21:46:02.125 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:02.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.313+0000 7f5616e00640 1 Processor -- start 2024-04-26T21:46:02.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.314+0000 7f5616e00640 1 -- start start 2024-04-26T21:46:02.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f561015f420 0x7f56100d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56100d3680 0x7f56100d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f56100cfb90 0x7f56100d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f56100d2c70 con 0x7f56100d3680 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f56100d3480 con 0x7f56100cfb90 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.315+0000 7f5616e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f561015f800 con 0x7f561015f420 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56100d3680 0x7f56100d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56100d3680 0x7f56100d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51870/0 (socket says 172.21.15.106:51870) 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 -- 172.21.15.106:0/1973301931 learned_addr learned my addr 172.21.15.106:0/1973301931 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560fe00640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f561015f420 0x7f56100d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f5614e00640 1 --2- 172.21.15.106:0/1973301931 >> v2:172.21.15.204:3300/0 conn(0x7f56100cfb90 0x7f56100d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 -- 172.21.15.106:0/1973301931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f561015f420 msgr2=0x7f56100d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f561015f420 0x7f56100d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 -- 172.21.15.106:0/1973301931 >> v2:172.21.15.204:3300/0 conn(0x7f56100cfb90 msgr2=0x7f56100d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 --2- 172.21.15.106:0/1973301931 >> v2:172.21.15.204:3300/0 conn(0x7f56100cfb90 0x7f56100d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.316+0000 7f560f400640 1 -- 172.21.15.106:0/1973301931 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56100d4020 con 0x7f56100d3680 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.317+0000 7f560f400640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f56100d3ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f561008f040 tx=0x7f55fc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec5bc9294cb804 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.317+0000 7f560ea00640 1 -- 172.21.15.106:0/1973301931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc036070 con 0x7f56100d3680 2024-04-26T21:46:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.317+0000 7f560ea00640 1 -- 172.21.15.106:0/1973301931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc031600 con 0x7f56100d3680 2024-04-26T21:46:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.317+0000 7f560ea00640 1 -- 172.21.15.106:0/1973301931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0319d0 con 0x7f56100d3680 2024-04-26T21:46:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 -- 172.21.15.106:0/1973301931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 msgr2=0x7f56100d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f56100d3ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f561008f040 tx=0x7f55fc02a610 comp rx=0 tx=0).stop 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 -- 172.21.15.106:0/1973301931 shutdown_connections 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 --2- 172.21.15.106:0/1973301931 >> v2:172.21.15.204:3300/0 conn(0x7f56100cfb90 0x7f56100d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f56100d3ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 --2- 172.21.15.106:0/1973301931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f561015f420 0x7f56100d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 -- 172.21.15.106:0/1973301931 >> 172.21.15.106:0/1973301931 conn(0x7f561009e3e0 msgr2=0x7f56100d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 -- 172.21.15.106:0/1973301931 shutdown_connections 2024-04-26T21:46:02.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.318+0000 7f5616e00640 1 -- 172.21.15.106:0/1973301931 wait complete. 2024-04-26T21:46:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.320+0000 7f5616e00640 1 Processor -- start 2024-04-26T21:46:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.320+0000 7f5616e00640 1 -- start start 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56100cfb90 0x7f56100b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561015f420 0x7f56101f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56100c8d20 con 0x7f56100d3680 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56100d31a0 con 0x7f561015f420 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f5616e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56100c9000 con 0x7f56100cfb90 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f560f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f560f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51884/0 (socket says 172.21.15.106:51884) 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.321+0000 7f560f400640 1 -- 172.21.15.106:0/2254165662 learned_addr learned my addr 172.21.15.106:0/2254165662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560fe00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56100cfb90 0x7f56100b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f5614e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561015f420 0x7f56101f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 -- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56100cfb90 msgr2=0x7f56100b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56100cfb90 0x7f56100b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 -- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561015f420 msgr2=0x7f56101f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561015f420 0x7f56101f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 -- 172.21.15.106:0/2254165662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56101f9d80 con 0x7f56100d3680 2024-04-26T21:46:02.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f560f400640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f55fc034040 tx=0x7f55fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.322+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc041020 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.323+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc007820 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.323+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc036040 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.323+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56100cc530 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.323+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56100ccaf0 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.324+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5610069540 con 0x7f56100d3680 2024-04-26T21:46:02.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.325+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55fc044020 con 0x7f56100d3680 2024-04-26T21:46:02.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.325+0000 7f5607e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.326+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55fc0ac960 con 0x7f56100d3680 2024-04-26T21:46:02.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.326+0000 7f560fe00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.326+0000 7f560fe00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 0x7f55f006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5600006fd0 tx=0x7f5600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.330+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55fc04b030 con 0x7f56100d3680 2024-04-26T21:46:02.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.533+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56100bacd0 con 0x7f56100d3680 2024-04-26T21:46:02.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.536+0000 7f5607e00640 1 -- 172.21.15.106:0/2254165662 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f55fc07ad80 con 0x7f56100d3680 2024-04-26T21:46:02.536 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:02.536 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 msgr2=0x7f55f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 0x7f55f006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5600006fd0 tx=0x7f5600008040 comp rx=0 tx=0).stop 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 msgr2=0x7f5610153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f55fc034040 tx=0x7f55fc004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 shutdown_connections 2024-04-26T21:46:02.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561015f420 0x7f56101f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56100d3680 0x7f5610153840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 --2- 172.21.15.106:0/2254165662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56100cfb90 0x7f56100b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 >> 172.21.15.106:0/2254165662 conn(0x7f561009e3e0 msgr2=0x7f561006e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 shutdown_connections 2024-04-26T21:46:02.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.539+0000 7f5616e00640 1 -- 172.21.15.106:0/2254165662 wait complete. 2024-04-26T21:46:02.557 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T21:46:02.558 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:02.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.758+0000 7f2d44a00640 1 Processor -- start 2024-04-26T21:46:02.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.758+0000 7f2d44a00640 1 -- start start 2024-04-26T21:46:02.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.759+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 0x7f2d400add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.759+0000 7f2d44a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d400af9a0 con 0x7f2d400cd650 2024-04-26T21:46:02.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d44a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d400afc80 con 0x7f2d4015cdf0 2024-04-26T21:46:02.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d44a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d400ae2c0 con 0x7f2d4006ce80 2024-04-26T21:46:02.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d3e800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2d400cd650 0x7f2d400b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58798/0 (socket says 172.21.15.106:58798) 2024-04-26T21:46:02.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d3e800640 1 -- 172.21.15.106:0/1377146569 learned_addr learned my addr 172.21.15.106:0/1377146569 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d37e00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 0x7f2d400add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764082428 0 0) 0x7f2d400af9a0 con 0x7f2d400cd650 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.760+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d24003520 con 0x7f2d400cd650 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 521460812 0 0) 0x7f2d24003520 con 0x7f2d400cd650 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 msgr2=0x7f2d400add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 0x7f2d400add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 >> v1:172.21.15.204:6789/0 conn(0x7f2d4015cdf0 legacy=0x7f2d4006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d400aff60 con 0x7f2d400cd650 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2d380019f0 con 0x7f2d400cd650 2024-04-26T21:46:02.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 >> v1:172.21.15.106:6789/0 conn(0x7f2d400cd650 legacy=0x7f2d400b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d240039e0 0x7f2d24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d24004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 0x7f2d240050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2400b9c0 con 0x7f2d24004390 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2400bb40 con 0x7f2d240039e0 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2400bd40 con 0x7f2d4006ce80 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d37e00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d24004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.761+0000 7f2d3de00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d240039e0 0x7f2d24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.762+0000 7f2d37e00640 1 -- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 msgr2=0x7f2d240050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.762+0000 7f2d37e00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d4006ce80 0x7f2d240050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.762+0000 7f2d37e00640 1 -- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d240039e0 msgr2=0x7f2d24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.762+0000 7f2d37e00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d240039e0 0x7f2d24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.762+0000 7f2d37e00640 1 -- 172.21.15.106:0/1377146569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24003520 con 0x7f2d24004390 2024-04-26T21:46:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d37e00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d24004860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ac0 tx=0x7f2d24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c15190aac9a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28036070 con 0x7f2d24004390 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d28031400 con 0x7f2d24004390 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d3d400640 1 -- 172.21.15.106:0/1377146569 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28009690 con 0x7f2d24004390 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d44a00640 1 -- 172.21.15.106:0/1377146569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 msgr2=0x7f2d24004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d44a00640 1 --2- 172.21.15.106:0/1377146569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d24004860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ac0 tx=0x7f2d24005830 comp rx=0 tx=0).stop 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.763+0000 7f2d3e800640 1 -- 172.21.15.106:0/1377146569 reap_dead start 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.764+0000 7f2d44a00640 1 -- 172.21.15.106:0/1377146569 shutdown_connections 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.764+0000 7f2d44a00640 1 -- 172.21.15.106:0/1377146569 >> 172.21.15.106:0/1377146569 conn(0x7f2d400b2d50 msgr2=0x7f2d400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.764+0000 7f2d44a00640 1 -- 172.21.15.106:0/1377146569 shutdown_connections 2024-04-26T21:46:02.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.764+0000 7f2d44a00640 1 -- 172.21.15.106:0/1377146569 wait complete. 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.765+0000 7f2d44a00640 1 Processor -- start 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.765+0000 7f2d44a00640 1 -- start start 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d40153990 0x7f2d40153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 0x7f2d401443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4015d1d0 con 0x7f2d24004390 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d400b05b0 con 0x7f2d40143f50 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.766+0000 7f2d44a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4015d350 con 0x7f2d40153990 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51910/0 (socket says 172.21.15.106:51910) 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 -- 172.21.15.106:0/2332744319 learned_addr learned my addr 172.21.15.106:0/2332744319 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:02.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3e800640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 0x7f2d401443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d37e00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d40153990 0x7f2d40153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 -- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d40153990 msgr2=0x7f2d40153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d40153990 0x7f2d40153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 -- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 msgr2=0x7f2d401443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 0x7f2d401443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 -- 172.21.15.106:0/2332744319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d40154350 con 0x7f2d24004390 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3e800640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 0x7f2d401443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.767+0000 7f2d3de00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e800 tx=0x7f2d3000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.768+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d300050d0 con 0x7f2d24004390 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.768+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30018070 con 0x7f2d24004390 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.768+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000c6c0 con 0x7f2d24004390 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.768+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d40144e10 con 0x7f2d24004390 2024-04-26T21:46:02.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.768+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d401548e0 con 0x7f2d24004390 2024-04-26T21:46:02.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.769+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d401552d0 con 0x7f2d24004390 2024-04-26T21:46:02.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.770+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d3000c860 con 0x7f2d24004390 2024-04-26T21:46:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.771+0000 7f2d36000640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.771+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d3008ba90 con 0x7f2d24004390 2024-04-26T21:46:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.771+0000 7f2d37e00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.772+0000 7f2d37e00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ac0 tx=0x7f2d28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.775+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d30059e20 con 0x7f2d24004390 2024-04-26T21:46:02.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.970+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d40069540 con 0x7f2d24004390 2024-04-26T21:46:02.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.972+0000 7f2d36000640 1 -- 172.21.15.106:0/2332744319 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2d30059720 con 0x7f2d24004390 2024-04-26T21:46:02.972 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:02.972 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 msgr2=0x7f2d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ac0 tx=0x7f2d28008030 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 msgr2=0x7f2d401f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e800 tx=0x7f2d3000ecc0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 shutdown_connections 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d40143f50 0x7f2d401443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d40153990 0x7f2d40153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 --2- 172.21.15.106:0/2332744319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d24004390 0x7f2d401f1c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.974+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 >> 172.21.15.106:0/2332744319 conn(0x7f2d400b2d50 msgr2=0x7f2d4006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.975+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 shutdown_connections 2024-04-26T21:46:02.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:02.975+0000 7f2d44a00640 1 -- 172.21.15.106:0/2332744319 wait complete. 2024-04-26T21:46:02.993 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T21:46:02.994 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T21:46:02.994 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T21:46:02.995 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T21:46:02.995 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T21:46:02.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T21:46:02.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T21:46:02.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T21:46:03.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.273+0000 7f242d600640 1 Processor -- start 2024-04-26T21:46:03.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.274+0000 7f242d600640 1 -- start start 2024-04-26T21:46:03.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.276+0000 7f242d600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 0x7f2428072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.276+0000 7f242d600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24280aee10 0x7f242815b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.277+0000 7f242d600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242809eec0 con 0x7f2428070580 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.277+0000 7f242d600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f242815b6a0 con 0x7f24280d1350 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.277+0000 7f242d600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2428073950 con 0x7f24280aee10 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f2426000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2428070580 0x7f24280ae700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:38278/0 (socket says 172.21.15.106:38278) 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f2426000640 1 -- 172.21.15.106:0/460712999 learned_addr learned my addr 172.21.15.106:0/460712999 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f2427400640 1 --2- 172.21.15.106:0/460712999 >> v2:172.21.15.204:3300/0 conn(0x7f24280aee10 0x7f242815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f2426a00640 1 --2- 172.21.15.106:0/460712999 >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 0x7f2428072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f008ee00640 1 Processor -- start 2024-04-26T21:46:03.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.278+0000 7f008ee00640 1 -- start start 2024-04-26T21:46:03.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7f2427400640 1 -- 172.21.15.106:0/460712999 >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 msgr2=0x7f2428072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7fe05cc00640 1 Processor -- start 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7fe05cc00640 1 -- start start 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7fe05cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0500eb060 0x7fe050055300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7f008ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 0x7f00881622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7f008ee00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880d3470 con 0x7f00880d1b90 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7f008ee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880d3750 con 0x7f00880d1f70 2024-04-26T21:46:03.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7f008ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00880d3210 con 0x7f00880b6350 2024-04-26T21:46:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7fe05cc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0500387e0 con 0x7fe0500061a0 2024-04-26T21:46:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7fe05cc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe050056d90 con 0x7fe0500eb440 2024-04-26T21:46:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.281+0000 7fe05cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe050057e30 con 0x7fe0500eb060 2024-04-26T21:46:03.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7f2427400640 1 --2- 172.21.15.106:0/460712999 >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 0x7f2428072a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.282+0000 7f0087e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f00880d1b90 0x7f00880d2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58828/0 (socket says 172.21.15.106:58828) 2024-04-26T21:46:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.282+0000 7f0087e00640 1 -- 172.21.15.106:0/3904523035 learned_addr learned my addr 172.21.15.106:0/3904523035 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.282+0000 7fe057e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe0500061a0 0x7fe050006630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58814/0 (socket says 172.21.15.106:58814) 2024-04-26T21:46:03.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.282+0000 7fe057e00640 1 -- 172.21.15.106:0/1015153677 learned_addr learned my addr 172.21.15.106:0/1015153677 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7f2427400640 1 -- 172.21.15.106:0/460712999 >> v1:172.21.15.204:6789/0 conn(0x7f2428070580 legacy=0x7f24280ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.279+0000 7f2427400640 1 -- 172.21.15.106:0/460712999 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280730b0 con 0x7f24280aee10 2024-04-26T21:46:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.280+0000 7f2426a00640 1 --2- 172.21.15.106:0/460712999 >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 0x7f2428072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.282+0000 7f0087400640 1 --2- 172.21.15.106:0/3904523035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 0x7f00881622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f2427400640 1 --2- 172.21.15.106:0/460712999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f242815b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24280b03a0 tx=0x7f241402a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e0c8a1d7a65701 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe057400640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0500eb060 0x7fe050055300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627824511 0 0) 0x7fe058050c30 con 0x7fe0500061a0 2024-04-26T21:46:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe040003520 con 0x7fe0500061a0 2024-04-26T21:46:03.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f2425600640 1 -- 172.21.15.106:0/460712999 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414037070 con 0x7f24280aee10 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f0087400640 1 -- 172.21.15.106:0/3904523035 >> v1:172.21.15.204:6789/0 conn(0x7f00880d1f70 legacy=0x7f00880d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f0087400640 1 -- 172.21.15.106:0/3904523035 >> v1:172.21.15.106:6789/0 conn(0x7f00880d1b90 legacy=0x7f00880d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f0087400640 1 -- 172.21.15.106:0/3904523035 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00880d3a30 con 0x7f00880b6350 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3502164999 0 0) 0x7fe040003520 con 0x7fe0500061a0 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0500eb060 msgr2=0x7fe050055300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0500eb060 0x7fe050055300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 >> v1:172.21.15.204:6789/0 conn(0x7fe0500eb440 legacy=0x7fe0500eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe050057070 con 0x7fe0500061a0 2024-04-26T21:46:03.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.283+0000 7f2425600640 1 -- 172.21.15.106:0/460712999 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414032440 con 0x7f24280aee10 2024-04-26T21:46:03.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 -- 172.21.15.106:0/460712999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 msgr2=0x7f242815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 --2- 172.21.15.106:0/460712999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f242815b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24280b03a0 tx=0x7f241402a150 comp rx=0 tx=0).stop 2024-04-26T21:46:03.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fe05806a4b0 con 0x7fe0500061a0 2024-04-26T21:46:03.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 >> v1:172.21.15.106:6789/0 conn(0x7fe0500061a0 legacy=0x7fe050006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 -- 172.21.15.106:0/460712999 shutdown_connections 2024-04-26T21:46:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 --2- 172.21.15.106:0/460712999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f242815b160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 --2- 172.21.15.106:0/460712999 >> v2:172.21.15.106:3300/0 conn(0x7f24280d1350 0x7f2428072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7f242d600640 1 -- 172.21.15.106:0/460712999 >> 172.21.15.106:0/460712999 conn(0x7f24280b3240 msgr2=0x7f24280d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0400039e0 0x7fe040003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe040004800 0x7fe040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0400051c0 0x7fe04000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe04000c090 con 0x7fe0400051c0 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe04000c210 con 0x7fe040004800 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.284+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe04000c410 con 0x7fe0400039e0 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.285+0000 7fe057e00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0400051c0 0x7fe04000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe056a00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe040004800 0x7fe040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057400640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0400039e0 0x7fe040003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7f0087400640 1 --2- 172.21.15.106:0/3904523035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 0x7f00881622a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f007c009750 tx=0x7f007c02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15233ec2a54e030e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057e00640 1 -- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0400039e0 msgr2=0x7fe040003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057e00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0400039e0 0x7fe040003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057e00640 1 -- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe040004800 msgr2=0x7fe040004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057e00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe040004800 0x7fe040004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7fe057e00640 1 -- 172.21.15.106:0/1015153677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040003520 con 0x7fe0400051c0 2024-04-26T21:46:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.287+0000 7f0086a00640 1 -- 172.21.15.106:0/3904523035 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c037070 con 0x7f00880b6350 2024-04-26T21:46:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.287+0000 7f0086a00640 1 -- 172.21.15.106:0/3904523035 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007c02adb0 con 0x7f00880b6350 2024-04-26T21:46:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.287+0000 7f0086a00640 1 -- 172.21.15.106:0/3904523035 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c032660 con 0x7f00880b6350 2024-04-26T21:46:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7fe057e00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0400051c0 0x7fe04000b9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe05806bba0 tx=0x7fe040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1941a749e59594f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580b4070 con 0x7fe0400051c0 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7f008ee00640 1 -- 172.21.15.106:0/3904523035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 msgr2=0x7f00881622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7f008ee00640 1 --2- 172.21.15.106:0/3904523035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 0x7f00881622a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f007c009750 tx=0x7f007c02a0a0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7f242d600640 1 -- 172.21.15.106:0/460712999 shutdown_connections 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.286+0000 7f242d600640 1 -- 172.21.15.106:0/460712999 wait complete. 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0580a7c40 con 0x7fe0400051c0 2024-04-26T21:46:03.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.288+0000 7fe056000640 1 -- 172.21.15.106:0/1015153677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058069550 con 0x7fe0400051c0 2024-04-26T21:46:03.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7f242d600640 1 Processor -- start 2024-04-26T21:46:03.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7f008ee00640 1 -- 172.21.15.106:0/3904523035 shutdown_connections 2024-04-26T21:46:03.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7f008ee00640 1 --2- 172.21.15.106:0/3904523035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880b6350 0x7f00881622a0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f007c009750 tx=0x7f007c02a0a0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fe056a00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe040004800 0x7fe040004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7f008ee00640 1 -- 172.21.15.106:0/3904523035 >> 172.21.15.106:0/3904523035 conn(0x7f008809e200 msgr2=0x7f00880d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fce47a00640 1 Processor -- start 2024-04-26T21:46:03.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fe05cc00640 1 -- 172.21.15.106:0/1015153677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0400051c0 msgr2=0x7fe04000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fe05cc00640 1 --2- 172.21.15.106:0/1015153677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0400051c0 0x7fe04000b9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe05806bba0 tx=0x7fe040005830 comp rx=0 tx=0).stop 2024-04-26T21:46:03.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fe057e00640 1 -- 172.21.15.106:0/1015153677 reap_dead start 2024-04-26T21:46:03.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7fce47a00640 1 -- start start 2024-04-26T21:46:03.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.290+0000 7fe05cc00640 1 -- 172.21.15.106:0/1015153677 shutdown_connections 2024-04-26T21:46:03.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.290+0000 7fe05cc00640 1 -- 172.21.15.106:0/1015153677 >> 172.21.15.106:0/1015153677 conn(0x7fe0500559e0 msgr2=0x7fe050055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.290+0000 7fe05cc00640 1 -- 172.21.15.106:0/1015153677 shutdown_connections 2024-04-26T21:46:03.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.290+0000 7fe05cc00640 1 -- 172.21.15.106:0/1015153677 wait complete. 2024-04-26T21:46:03.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.289+0000 7f242d600640 1 -- start start 2024-04-26T21:46:03.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.290+0000 7f242d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 0x7f24281546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.291+0000 7f008ee00640 1 -- 172.21.15.106:0/3904523035 shutdown_connections 2024-04-26T21:46:03.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.291+0000 7f242d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f2428150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.291+0000 7f008ee00640 1 -- 172.21.15.106:0/3904523035 wait complete. 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.291+0000 7f2426a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 0x7f24281546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.291+0000 7f2426a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 0x7f24281546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39454/0 (socket says 172.21.15.106:39454) 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.292+0000 7f2426000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f2428150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.292+0000 7f2426000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f2428150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37404/0 (socket says 172.21.15.106:37404) 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.292+0000 7f242d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24280d1350 0x7f2428150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.292+0000 7f2426a00640 1 -- 172.21.15.106:0/3253873863 learned_addr learned my addr 172.21.15.106:0/3253873863 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.292+0000 7fce47a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce400d6100 0x7fce400bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fce47a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 0x7fce40167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fce47a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce40073ff0 con 0x7fce40073af0 2024-04-26T21:46:03.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24280b4670 con 0x7f24280d1350 2024-04-26T21:46:03.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2428073670 con 0x7f24280aee10 2024-04-26T21:46:03.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fe05cc00640 1 Processor -- start 2024-04-26T21:46:03.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24280b4b90 con 0x7f2428070580 2024-04-26T21:46:03.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fe05cc00640 1 -- start start 2024-04-26T21:46:03.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fce47a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fce40070650 con 0x7fce400d6100 2024-04-26T21:46:03.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7fce47a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fce400d7590 con 0x7fce400d8070 2024-04-26T21:46:03.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7f2427400640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24280d1350 0x7f2428150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fce45a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 0x7fce40167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fa9fc200640 1 Processor -- start 2024-04-26T21:46:03.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fce45000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce400d6100 0x7fce400bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fce45a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 0x7fce40167bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37408/0 (socket says 172.21.15.106:37408) 2024-04-26T21:46:03.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fce45a00640 1 -- 172.21.15.106:0/1522814187 learned_addr learned my addr 172.21.15.106:0/1522814187 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7fa9fc200640 1 -- start start 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7f2426a00640 1 -- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 msgr2=0x7f2428150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7f2426a00640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f2428150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7f2426a00640 1 -- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24280d1350 msgr2=0x7f2428150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7f2426a00640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24280d1350 0x7f2428150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.294+0000 7f2426a00640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24281535d0 con 0x7f2428070580 2024-04-26T21:46:03.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.293+0000 7f008ee00640 1 Processor -- start 2024-04-26T21:46:03.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7f2426000640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24280aee10 0x7f2428150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:03.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fce45000640 1 -- 172.21.15.106:0/1522814187 >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 msgr2=0x7fce40167bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fe05cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 0x7fe050050130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7f2427400640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24280d1350 0x7f2428150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7f2426a00640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 0x7f24281546f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2420007ef0 tx=0x7f242000d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fe05cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe050173a40 0x7fe050051900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fe05cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe050051e40 0x7fe05004cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fce45000640 1 --2- 172.21.15.106:0/1522814187 >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 0x7fce40167bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fce45000640 1 -- 172.21.15.106:0/1522814187 >> v1:172.21.15.106:6789/0 conn(0x7fce40073af0 legacy=0x7fce40071ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:46:03.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.295+0000 7fce45000640 1 -- 172.21.15.106:0/1522814187 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce400d7870 con 0x7fce400d6100 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe05cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe0500579a0 con 0x7fe05004bc80 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9fc200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 0x7fa9ec03a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9fc200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa9ec00ab40 0x7fa9ec009d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9fc200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ec03a6b0 con 0x7fa9ec0516e0 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9fc200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec055920 con 0x7fa9ec00ab40 2024-04-26T21:46:03.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9fc200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec009b50 con 0x7fa9ec0e1310 2024-04-26T21:46:03.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7f008ee00640 1 -- start start 2024-04-26T21:46:03.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fa9f9800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 0x7fa9ec03a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fa9f9800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 0x7fa9ec03a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39484/0 (socket says 172.21.15.106:39484) 2024-04-26T21:46:03.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fa9f9800640 1 -- 172.21.15.106:0/3212935282 learned_addr learned my addr 172.21.15.106:0/3212935282 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242001a070 con 0x7f2428070580 2024-04-26T21:46:03.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fa9f8e00640 1 --2- 172.21.15.106:0/3212935282 >> v2:172.21.15.204:3300/0 conn(0x7fa9ec00ab40 0x7fa9ec009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24281538b0 con 0x7f2428070580 2024-04-26T21:46:03.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe05cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe050057630 con 0x7fe050173a40 2024-04-26T21:46:03.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe05cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe0500e1e10 con 0x7fe050051e40 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe057400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 0x7fe050050130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe056a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe050173a40 0x7fe050051900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe057400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 0x7fe050050130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51964/0 (socket says 172.21.15.106:51964) 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7f584dc00640 1 Processor -- start 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe057400640 1 -- 172.21.15.106:0/816600767 learned_addr learned my addr 172.21.15.106:0/816600767 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.296+0000 7fe057e00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe050051e40 0x7fe05004cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 -- 172.21.15.106:0/816600767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe050051e40 msgr2=0x7fe05004cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe050051e40 0x7fe05004cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 -- 172.21.15.106:0/816600767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe050173a40 msgr2=0x7fe050051900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe050173a40 0x7fe050051900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0500526b0 con 0x7fe05004bc80 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fe057400640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 0x7fe050050130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe04400f560 tx=0x7fe04400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7fce45000640 1 --2- 172.21.15.106:0/1522814187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce400bc3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce40069000 tx=0x7fce2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62617c75d2dfc79f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2428153e70 con 0x7f2428070580 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.297+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2420003b90 con 0x7f2428070580 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004030 con 0x7f2428070580 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fce3fe00640 1 -- 172.21.15.106:0/1522814187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c037070 con 0x7fce400d6100 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fce3fe00640 1 -- 172.21.15.106:0/1522814187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c02ae10 con 0x7fce400d6100 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fce3fe00640 1 -- 172.21.15.106:0/1522814187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c032660 con 0x7fce400d6100 2024-04-26T21:46:03.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7f584dc00640 1 -- start start 2024-04-26T21:46:03.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f3e00640 1 -- 172.21.15.106:0/3212935282 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602916439 0 0) 0x7fa9ec03a6b0 con 0x7fa9ec0516e0 2024-04-26T21:46:03.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f3e00640 1 -- 172.21.15.106:0/3212935282 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9dc003520 con 0x7fa9ec0516e0 2024-04-26T21:46:03.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7f008ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 0x7f008814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7f008ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880d1f70 0x7f00881f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f008ee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881f7ab0 0x7f008815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 -- 172.21.15.106:0/1522814187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 msgr2=0x7fce400bc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2400001030 con 0x7f2428070580 2024-04-26T21:46:03.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044019030 con 0x7fe05004bc80 2024-04-26T21:46:03.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 --2- 172.21.15.106:0/1522814187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce400bc3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce40069000 tx=0x7fce2c02a100 comp rx=0 tx=0).stop 2024-04-26T21:46:03.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe044004020 con 0x7fe05004bc80 2024-04-26T21:46:03.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044004410 con 0x7fe05004bc80 2024-04-26T21:46:03.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe050052990 con 0x7fe05004bc80 2024-04-26T21:46:03.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe050052f50 con 0x7fe05004bc80 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f008ee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00880b7d80 con 0x7f00881f7ab0 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f008ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00880cfad0 con 0x7f00880d1b90 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f008ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00880b7f00 con 0x7f00880d1f70 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f0087e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 0x7f008814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f0087e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 0x7f008814f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37430/0 (socket says 172.21.15.106:37430) 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f0087e00640 1 -- 172.21.15.106:0/85379341 learned_addr learned my addr 172.21.15.106:0/85379341 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f0087400640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880d1f70 0x7f00881f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 -- 172.21.15.106:0/1522814187 shutdown_connections 2024-04-26T21:46:03.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 --2- 172.21.15.106:0/1522814187 >> v2:172.21.15.204:3300/0 conn(0x7fce400d8070 0x7fce40167bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 --2- 172.21.15.106:0/1522814187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce400bc3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7fce47a00640 1 -- 172.21.15.106:0/1522814187 >> 172.21.15.106:0/1522814187 conn(0x7fce400bb0d0 msgr2=0x7fce400bb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f8e00640 1 -- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 msgr2=0x7fa9ec03a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f8e00640 1 --2- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 0x7fa9ec03a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f8e00640 1 -- 172.21.15.106:0/3212935282 >> v1:172.21.15.204:6789/0 conn(0x7fa9ec0516e0 legacy=0x7fa9ec055db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.298+0000 7fa9f8e00640 1 -- 172.21.15.106:0/3212935282 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec054d10 con 0x7fa9ec00ab40 2024-04-26T21:46:03.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.299+0000 7f3424a00640 1 Processor -- start 2024-04-26T21:46:03.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f008ce00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881f7ab0 0x7f008815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7fce47a00640 1 -- 172.21.15.106:0/1522814187 shutdown_connections 2024-04-26T21:46:03.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7fce47a00640 1 -- 172.21.15.106:0/1522814187 wait complete. 2024-04-26T21:46:03.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7fa9f8e00640 1 --2- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec009d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00097a0 tx=0x7fa9e002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e561898937b3110 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f3424a00640 1 -- start start 2024-04-26T21:46:03.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe034001030 con 0x7fe05004bc80 2024-04-26T21:46:03.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f0087e00640 1 -- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880d1f70 msgr2=0x7f00881f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f0087e00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00880d1f70 0x7f00881f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f0087e00640 1 -- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881f7ab0 msgr2=0x7f008815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f584dc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 0x7f584003a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f584dc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f584000a460 0x7f5840055db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f584dc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584003a6b0 con 0x7f584000ab40 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f584dc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5840055920 con 0x7f58400e1310 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.300+0000 7f584dc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5840054d10 con 0x7f584000a460 2024-04-26T21:46:03.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7fa9f3e00640 1 -- 172.21.15.106:0/3212935282 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0037070 con 0x7fa9ec00ab40 2024-04-26T21:46:03.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f0087e00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881f7ab0 0x7f008815cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f0087e00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00881f80e0 con 0x7f00880d1b90 2024-04-26T21:46:03.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7fa9f3e00640 1 -- 172.21.15.106:0/3212935282 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0032440 con 0x7fa9ec00ab40 2024-04-26T21:46:03.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7fa9f3e00640 1 -- 172.21.15.106:0/3212935282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e002ae60 con 0x7fa9ec00ab40 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f3424a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 0x7f342015f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f3424a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34200d5d90 0x7f34200b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f3424a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34200b5d30 con 0x7f34200d34a0 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f3424a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f34200b6010 con 0x7f34200d5d90 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f3424a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34200b62f0 con 0x7f342015ef60 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f008ce00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881f7ab0 0x7f008815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:03.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f0087e00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 0x7f008814f1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f007400c480 tx=0x7f007400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f584d200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f584000a460 0x7f5840055db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f584d200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f584000a460 0x7f5840055db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37436/0 (socket says 172.21.15.106:37436) 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.301+0000 7f584d200640 1 -- 172.21.15.106:0/4051902544 learned_addr learned my addr 172.21.15.106:0/4051902544 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f5847e00640 1 --2- 172.21.15.106:0/4051902544 >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 0x7f584003a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f341de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 0x7f342015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f341d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34200d5d90 0x7f34200b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f341de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 0x7f342015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39502/0 (socket says 172.21.15.106:39502) 2024-04-26T21:46:03.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f341de00640 1 -- 172.21.15.106:0/482668561 learned_addr learned my addr 172.21.15.106:0/482668561 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7fa9fc200640 1 -- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec00ab40 msgr2=0x7fa9ec009d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec009d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00097a0 tx=0x7fa9e002a150 comp rx=0 tx=0).stop 2024-04-26T21:46:03.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7fce47a00640 1 Processor -- start 2024-04-26T21:46:03.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074017070 con 0x7f00880d1b90 2024-04-26T21:46:03.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0074005ce0 con 0x7f00880d1b90 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.302+0000 7fce47a00640 1 -- start start 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f584d200640 1 -- 172.21.15.106:0/4051902544 >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 msgr2=0x7f584003a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f584d200640 1 --2- 172.21.15.106:0/4051902544 >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 0x7f584003a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f584d200640 1 -- 172.21.15.106:0/4051902544 >> v1:172.21.15.106:6789/0 conn(0x7f584000ab40 legacy=0x7f5840009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f584d200640 1 -- 172.21.15.106:0/4051902544 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5840054ff0 con 0x7f584000a460 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00881f83c0 con 0x7f00880d1b90 2024-04-26T21:46:03.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00881f8980 con 0x7f00880d1b90 2024-04-26T21:46:03.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074007290 con 0x7f00880d1b90 2024-04-26T21:46:03.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fce40073af0 0x7fce400a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce401eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fb0ae200640 1 Processor -- start 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00880693c0 con 0x7f00880d1b90 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce400d8070 0x7fce4015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce400d64e0 con 0x7fce400d6100 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce400bc910 con 0x7fce400d8070 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce47a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fce400d6660 con 0x7fce40073af0 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce37e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce401eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce37e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce401eef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52004/0 (socket says 172.21.15.106:52004) 2024-04-26T21:46:03.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fce37e00640 1 -- 172.21.15.106:0/3984117662 learned_addr learned my addr 172.21.15.106:0/3984117662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fb0ae200640 1 -- start start 2024-04-26T21:46:03.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce45000640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fce40073af0 0x7fce400a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fa9fc200640 1 -- 172.21.15.106:0/3212935282 shutdown_connections 2024-04-26T21:46:03.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec009d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3212935282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0e1310 0x7fa9ec03a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7fa9fc200640 1 -- 172.21.15.106:0/3212935282 >> 172.21.15.106:0/3212935282 conn(0x7fa9ec056cd0 msgr2=0x7fa9ec0570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2420005d10 con 0x7f2428070580 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f241ea00640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f240806c9c0 0x7f240806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f242008c410 con 0x7f2428070580 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7f241ea00640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f2408074be0 0x7f2408077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2420023e50 con 0x7f2408074be0 2024-04-26T21:46:03.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f242008c800 con 0x7f2428070580 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe04401c070 con 0x7fe05004bc80 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce37e00640 1 -- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fce40073af0 msgr2=0x7fce400a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce37e00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fce40073af0 0x7fce400a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce37e00640 1 -- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce400d8070 msgr2=0x7fce4015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce45a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce400d8070 0x7fce4015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.303+0000 7f5847e00640 1 --2- 172.21.15.106:0/4051902544 >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 0x7f584003a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fa9fc200640 1 -- 172.21.15.106:0/3212935282 shutdown_connections 2024-04-26T21:46:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f341d400640 1 -- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 msgr2=0x7f342015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f341d400640 1 --2- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 0x7f342015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f341d400640 1 -- 172.21.15.106:0/482668561 >> v1:172.21.15.204:6789/0 conn(0x7f34200d34a0 legacy=0x7f3420162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.304+0000 7f341d400640 1 -- 172.21.15.106:0/482668561 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34200b65d0 con 0x7f34200d5d90 2024-04-26T21:46:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7f2426000640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f240806c9c0 0x7f240806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce37e00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce400d8070 0x7fce4015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.305+0000 7fce37e00640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce401ed740 con 0x7fce400d6100 2024-04-26T21:46:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fce45a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce400d8070 0x7fce4015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fa9fc200640 1 -- 172.21.15.106:0/3212935282 wait complete. 2024-04-26T21:46:03.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7f2427400640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f2408074be0 0x7f2408077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7f584d200640 1 --2- 172.21.15.106:0/4051902544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000a460 0x7f5840055db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f584000fb80 tx=0x7f583c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67bde833aab92c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fce37e00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce401eef30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b9e0 tx=0x7fce3000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fe04b400640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe03806c9c0 0x7fe03806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fb0ae200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0a80d5d90 0x7fb0a815f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fb0ae200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a80b5d30 con 0x7fb0a815ef60 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fb0ae200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a80b6010 con 0x7fb0a80d34a0 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.306+0000 7fb0ae200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a80b62f0 con 0x7fb0a80d5d90 2024-04-26T21:46:03.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe04408c770 con 0x7fe05004bc80 2024-04-26T21:46:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fe056a00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe03806c9c0 0x7fe03806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fb0a7400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb0a815ef60 0x7fb0a80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58848/0 (socket says 172.21.15.106:58848) 2024-04-26T21:46:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30019070 con 0x7fce400d6100 2024-04-26T21:46:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fe04b400640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7fe038074be0 0x7fe038077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3000cba0 con 0x7fce400d6100 2024-04-26T21:46:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30014440 con 0x7fce400d6100 2024-04-26T21:46:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fb0a7400640 1 -- 172.21.15.106:0/340967632 learned_addr learned my addr 172.21.15.106:0/340967632 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fb0a6a00640 1 --2- 172.21.15.106:0/340967632 >> v2:172.21.15.106:3300/0 conn(0x7fb0a80d5d90 0x7fb0a815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce401ed460 con 0x7fce400d6100 2024-04-26T21:46:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce401efc50 con 0x7fce400d6100 2024-04-26T21:46:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fb0a6a00640 1 -- 172.21.15.106:0/340967632 >> v1:172.21.15.204:6789/0 conn(0x7fb0a80d34a0 legacy=0x7fb0a80d01d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:46:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7f2427400640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f2408074be0 0x7f2408077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7f2426000640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f240806c9c0 0x7f240806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f242815a590 tx=0x7f2418007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe044001d30 con 0x7fe038074be0 2024-04-26T21:46:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.307+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe04408cb60 con 0x7fe05004bc80 2024-04-26T21:46:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fa9fc200640 1 Processor -- start 2024-04-26T21:46:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fe057e00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7fe038074be0 0x7fe038077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fa9fc200640 1 -- start start 2024-04-26T21:46:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== osd.0 v2:172.21.15.106:6800/2200991001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2420023e50 con 0x7f2408074be0 2024-04-26T21:46:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f5846a00640 1 -- 172.21.15.106:0/4051902544 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c037070 con 0x7f584000a460 2024-04-26T21:46:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f5846a00640 1 -- 172.21.15.106:0/4051902544 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583c032440 con 0x7f584000a460 2024-04-26T21:46:03.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f5846a00640 1 -- 172.21.15.106:0/4051902544 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c032810 con 0x7f584000a460 2024-04-26T21:46:03.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7fe056a00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe03806c9c0 0x7fe03806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe0500ebf60 tx=0x7fe04c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f341d400640 1 --2- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34200d5d90 0x7f34200b5450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f340c0097a0 tx=0x7f340c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b5df6febeb4d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7fe057e00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7fe038074be0 0x7fe038077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f007401a070 con 0x7f00880d1b90 2024-04-26T21:46:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec04b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 -- 172.21.15.106:0/4051902544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000a460 msgr2=0x7f5840055db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 --2- 172.21.15.106:0/4051902544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000a460 0x7f5840055db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f584000fb80 tx=0x7f583c02a210 comp rx=0 tx=0).stop 2024-04-26T21:46:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f341ca00640 1 -- 172.21.15.106:0/482668561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c037070 con 0x7f34200d5d90 2024-04-26T21:46:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0516e0 0x7fa9ec0e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f341ca00640 1 -- 172.21.15.106:0/482668561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c02ada0 con 0x7f34200d5d90 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec0e1310 0x7fa9ec0e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec0555b0 con 0x7fa9ec00ab40 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec0552d0 con 0x7fa9ec0e1310 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7fa9fc200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec0548a0 con 0x7fa9ec0516e0 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 -- 172.21.15.106:0/4051902544 shutdown_connections 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 --2- 172.21.15.106:0/4051902544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000a460 0x7f5840055db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 --2- 172.21.15.106:0/4051902544 >> v2:172.21.15.106:3300/0 conn(0x7f58400e1310 0x7f584003a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 -- 172.21.15.106:0/4051902544 >> 172.21.15.106:0/4051902544 conn(0x7f5840056cd0 msgr2=0x7f58400570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 -- 172.21.15.106:0/4051902544 shutdown_connections 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.310+0000 7f584dc00640 1 -- 172.21.15.106:0/4051902544 wait complete. 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fb0a6000640 1 -- 172.21.15.106:0/340967632 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318594236 0 0) 0x7fb0a80b5d30 con 0x7fb0a815ef60 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fb0a6a00640 1 -- 172.21.15.106:0/340967632 >> v1:172.21.15.106:6789/0 conn(0x7fb0a815ef60 legacy=0x7fb0a80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.308+0000 7fb0a6a00640 1 -- 172.21.15.106:0/340967632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a80b65d0 con 0x7fb0a80d5d90 2024-04-26T21:46:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fa9f9800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec04b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fa9fa200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec0e1310 0x7fa9ec0e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fa9fa200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec0e1310 0x7fa9ec0e40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37470/0 (socket says 172.21.15.106:37470) 2024-04-26T21:46:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== osd.1 v2:172.21.15.106:6804/1887774310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe044001d30 con 0x7fe038074be0 2024-04-26T21:46:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.309+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce40069540 con 0x7fce400d6100 2024-04-26T21:46:03.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fa9fa200640 1 -- 172.21.15.106:0/3512727915 learned_addr learned my addr 172.21.15.106:0/3512727915 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7f3424a00640 1 -- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34200d5d90 msgr2=0x7f34200b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7f3424a00640 1 --2- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34200d5d90 0x7f34200b5450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f340c0097a0 tx=0x7f340c02a110 comp rx=0 tx=0).stop 2024-04-26T21:46:03.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7f341ca00640 1 -- 172.21.15.106:0/482668561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c037070 con 0x7f34200d5d90 2024-04-26T21:46:03.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7fa9f8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0516e0 0x7fa9ec0e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.311+0000 7f007be00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f006806c940 0x7f006806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f007408b5a0 con 0x7f00880d1b90 2024-04-26T21:46:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fb0a6a00640 1 --2- 172.21.15.106:0/340967632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a815f340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb09402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4adf32bddf72ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f3424a00640 1 -- 172.21.15.106:0/482668561 shutdown_connections 2024-04-26T21:46:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fa9f9800640 1 -- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0516e0 msgr2=0x7fa9ec0e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f3424a00640 1 --2- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34200d5d90 0x7f34200b5450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fa9f9800640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec0516e0 0x7fa9ec0e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fa9f9800640 1 -- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec0e1310 msgr2=0x7fa9ec0e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fa9f9800640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec0e1310 0x7fa9ec0e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7fa9f9800640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec0cf740 con 0x7fa9ec00ab40 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f007be00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f0068074b40 0x7f0068076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f007401be50 con 0x7f0068074b40 2024-04-26T21:46:03.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.312+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f007408ba30 con 0x7f00880d1b90 2024-04-26T21:46:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f584dc00640 1 Processor -- start 2024-04-26T21:46:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f584dc00640 1 -- start start 2024-04-26T21:46:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f3424a00640 1 --2- 172.21.15.106:0/482668561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f342015ef60 0x7f342015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f3424a00640 1 -- 172.21.15.106:0/482668561 >> 172.21.15.106:0/482668561 conn(0x7f342009e3b0 msgr2=0x7f34200d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f0087400640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f006806c940 0x7f006806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f008ce00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f0068074b40 0x7f0068076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7fa9f9800640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec04b6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e850 tx=0x7fa9e800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7f3424a00640 1 -- 172.21.15.106:0/482668561 shutdown_connections 2024-04-26T21:46:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.313+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce3001c070 con 0x7fce400d6100 2024-04-26T21:46:03.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7fb0a6000640 1 -- 172.21.15.106:0/340967632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094037070 con 0x7fb0a80d5d90 2024-04-26T21:46:03.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7f584dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f584000a460 0x7f584004ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7f0087400640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f006806c940 0x7f006806ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f007c034000 tx=0x7f007c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8017070 con 0x7fa9ec00ab40 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e800c940 con 0x7fa9ec00ab40 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800ccf0 con 0x7fa9ec00ab40 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7f584dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 0x7f584004b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fce3d600640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fce2006c9c0 0x7fce2006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce3008b7a0 con 0x7fce400d6100 2024-04-26T21:46:03.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fce3d600640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7fce20074be0 0x7fce20077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce200776b0 con 0x7fce20074be0 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce30092050 con 0x7fce400d6100 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7f584dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58400e1310 0x7f584004d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7f584dc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58400548a0 con 0x7f58400e1310 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7f584dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58400555b0 con 0x7f584000ab40 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7f584dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5840054a20 con 0x7f584000a460 2024-04-26T21:46:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec0e4610 con 0x7fa9ec00ab40 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.315+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec0d7250 con 0x7fa9ec00ab40 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7fce45000640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fce2006c9c0 0x7fce2006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f5847400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 0x7f584004b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f5847400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 0x7f584004b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37474/0 (socket says 172.21.15.106:37474) 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f5847400640 1 -- 172.21.15.106:0/2517515788 learned_addr learned my addr 172.21.15.106:0/2517515788 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f584d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58400e1310 0x7f584004d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.314+0000 7f3424a00640 1 -- 172.21.15.106:0/482668561 wait complete. 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f3424a00640 1 Processor -- start 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7f3424a00640 1 -- start start 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7fb0a6000640 1 -- 172.21.15.106:0/340967632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb094032440 con 0x7fb0a80d5d90 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.316+0000 7fb0a6000640 1 -- 172.21.15.106:0/340967632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094032850 con 0x7fb0a80d5d90 2024-04-26T21:46:03.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7f5847e00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f584000a460 0x7f584004ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7fb0ae200640 1 -- 172.21.15.106:0/340967632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 msgr2=0x7fb0a815f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7fb0ae200640 1 --2- 172.21.15.106:0/340967632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a815f340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb09402a290 comp rx=0 tx=0).stop 2024-04-26T21:46:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7fce45a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7fce20074be0 0x7fce20077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7fce45000640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fce2006c9c0 0x7fce2006ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034000 tx=0x7fce2c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.317+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9ec0057e0 con 0x7fa9ec00ab40 2024-04-26T21:46:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.318+0000 7f3424a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34200d34a0 0x7f34200cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.318+0000 7f3424a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34200d5d90 0x7f34200b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.318+0000 7fb0ae200640 1 -- 172.21.15.106:0/340967632 shutdown_connections 2024-04-26T21:46:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.318+0000 7fb0ae200640 1 --2- 172.21.15.106:0/340967632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a815f340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.318+0000 7fb0ae200640 1 -- 172.21.15.106:0/340967632 >> 172.21.15.106:0/340967632 conn(0x7fb0a809e3b0 msgr2=0x7fb0a80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f3424a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342015ef60 0x7f34200c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f341d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34200d5d90 0x7f34200b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f584d200640 1 -- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f584000a460 msgr2=0x7f584004ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f584d200640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f584000a460 0x7f584004ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f341d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34200d5d90 0x7f34200b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52046/0 (socket says 172.21.15.106:52046) 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f3424a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34200b5b10 con 0x7f34200d5d90 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f3424a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34200b5990 con 0x7f342015ef60 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f3424a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34200d11e0 con 0x7f34200d34a0 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f341d400640 1 -- 172.21.15.106:0/1604965177 learned_addr learned my addr 172.21.15.106:0/1604965177 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7fb0ae200640 1 -- 172.21.15.106:0/340967632 shutdown_connections 2024-04-26T21:46:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7fb0ae200640 1 -- 172.21.15.106:0/340967632 wait complete. 2024-04-26T21:46:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f341de00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34200d34a0 0x7f34200cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f008ce00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f0068074b40 0x7f0068076f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.320+0000 7f341e800640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342015ef60 0x7f34200c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.320+0000 7fce45a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7fce20074be0 0x7fce20077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f584d200640 1 -- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 msgr2=0x7f584004b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f584d200640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 0x7f584004b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.319+0000 7f584d200640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584004bdd0 con 0x7f58400e1310 2024-04-26T21:46:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.320+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9e801a070 con 0x7fa9ec00ab40 2024-04-26T21:46:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f5847400640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f584000ab40 0x7f584004b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f341e800640 1 -- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34200d34a0 msgr2=0x7f34200cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f341e800640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34200d34a0 0x7f34200cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f341e800640 1 -- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34200d5d90 msgr2=0x7f34200b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f5847e00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f584000a460 0x7f584004ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7f341e800640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34200d5d90 0x7f34200b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.321+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== osd.3 v2:172.21.15.106:6816/56943883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce200776b0 con 0x7fce20074be0 2024-04-26T21:46:03.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7f341e800640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34200cbb30 con 0x7f342015ef60 2024-04-26T21:46:03.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7f584d200640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58400e1310 0x7f584004d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f583c031000 tx=0x7f583c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== osd.2 v2:172.21.15.106:6824/323734454 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f007401be50 con 0x7f0068074b40 2024-04-26T21:46:03.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7fb0ae200640 1 Processor -- start 2024-04-26T21:46:03.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c041030 con 0x7f58400e1310 2024-04-26T21:46:03.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7fb0ae200640 1 -- start start 2024-04-26T21:46:03.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584004d650 con 0x7f58400e1310 2024-04-26T21:46:03.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f341e800640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342015ef60 0x7f34200c94e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f341800c760 tx=0x7f341800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7fa9f1600640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9d406c940 0x7fa9d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584004c350 con 0x7f58400e1310 2024-04-26T21:46:03.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.322+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583c03d4e0 con 0x7f58400e1310 2024-04-26T21:46:03.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c037040 con 0x7f58400e1310 2024-04-26T21:46:03.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418017070 con 0x7f342015ef60 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3418003da0 con 0x7f342015ef60 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34200b8c40 con 0x7f342015ef60 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e808b930 con 0x7fa9ec00ab40 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fa9f1600640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa9d4074b60 0x7fa9d4076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9d4077630 con 0x7fa9d4074b60 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9e80902c0 con 0x7fa9ec00ab40 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fa9f8e00640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9d406c940 0x7fa9d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.327+0000 7fa9f8e00640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9d406c940 0x7fa9d406ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.323+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34200c8ce0 con 0x7f342015ef60 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.325+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180052b0 con 0x7f342015ef60 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.325+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34200b9160 con 0x7f342015ef60 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.327+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f341801a070 con 0x7f342015ef60 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0a80d34a0 0x7fb0a815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0a815ef60 0x7fb0a80b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a80d3ab0 con 0x7fb0a80d5d90 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a80d3930 con 0x7fb0a80d34a0 2024-04-26T21:46:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.324+0000 7fb0ae200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a80d3c30 con 0x7fb0a815ef60 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.325+0000 7fb0a6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.325+0000 7fb0a6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a80b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52056/0 (socket says 172.21.15.106:52056) 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.325+0000 7fb0a6a00640 1 -- 172.21.15.106:0/3334406474 learned_addr learned my addr 172.21.15.106:0/3334406474 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 -- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0a815ef60 msgr2=0x7fb0a80b9220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0a815ef60 0x7fb0a80b9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 -- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0a80d34a0 msgr2=0x7fb0a815f5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0a80d34a0 0x7fb0a815f5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a80b9850 con 0x7fb0a80d5d90 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.326+0000 7fb0a6a00640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a80b8ce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.327+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5824001030 con 0x7f58400e1310 2024-04-26T21:46:03.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.328+0000 7fa9fa200640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa9d4074b60 0x7fa9d4076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.327+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094042020 con 0x7fb0a80d5d90 2024-04-26T21:46:03.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.327+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0940308f0 con 0x7fb0a80d5d90 2024-04-26T21:46:03.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.328+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094037040 con 0x7fb0a80d5d90 2024-04-26T21:46:03.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.328+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a80b9b30 con 0x7fb0a80d5d90 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.328+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f583c044070 con 0x7f58400e1310 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7fa9fa200640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa9d4074b60 0x7fa9d4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.328+0000 7f3416000640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f340006c8f0 0x7f340006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f341808b600 con 0x7f342015ef60 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f3416000640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f3400074a90 0x7f3400076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f341800de50 con 0x7f3400074a90 2024-04-26T21:46:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f341808ba90 con 0x7f342015ef60 2024-04-26T21:46:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f341de00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f340006c8f0 0x7f340006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a80ba0f0 con 0x7fb0a80d5d90 2024-04-26T21:46:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.329+0000 7f583be00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f582806c8f0 0x7f582806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.330+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f583c0abe40 con 0x7f58400e1310 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.330+0000 7f583be00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7f5828074a90 0x7f5828076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.330+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f583c001d30 con 0x7f5828074a90 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.330+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f583c0ae460 con 0x7f58400e1310 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.330+0000 7f5847e00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f582806c8f0 0x7f582806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.331+0000 7f5847400640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7f5828074a90 0x7f5828076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.331+0000 7f341d400640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f3400074a90 0x7f3400076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.331+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== osd.5 v2:172.21.15.204:6816/885497049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9d4077630 con 0x7fa9d4074b60 2024-04-26T21:46:03.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.332+0000 7f341de00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f340006c8f0 0x7f340006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3410009bd0 tx=0x7f3410009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.332+0000 7f341d400640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f3400074a90 0x7f3400076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.332+0000 7f5847e00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f582806c8f0 0x7f582806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5834009ba0 tx=0x7f5834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.332+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0a8069540 con 0x7fb0a80d5d90 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.332+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb094045020 con 0x7fb0a80d5d90 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7fb09f400640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb08806c940 0x7fb08806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0940ac780 con 0x7fb0a80d5d90 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7fb09f400640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fb088074b60 0x7fb088076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb09402be50 con 0x7fb088074b60 2024-04-26T21:46:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0940b3050 con 0x7fb0a80d5d90 2024-04-26T21:46:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.333+0000 7f5847400640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7f5828074a90 0x7f5828076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.334+0000 7fb0a7400640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb08806c940 0x7fb08806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.334+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== osd.6 v2:172.21.15.204:6821/3330975067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f341800de50 con 0x7f3400074a90 2024-04-26T21:46:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.334+0000 7fb0a7e00640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fb088074b60 0x7fb088076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.335+0000 7fb0a7400640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb08806c940 0x7fb08806ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098007890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.335+0000 7fb0a7e00640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fb088074b60 0x7fb088076f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.336+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== osd.4 v2:172.21.15.204:6800/3289538728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f583c001d30 con 0x7f5828074a90 2024-04-26T21:46:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.338+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== osd.7 v2:172.21.15.204:6808/2746439320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb09402be50 con 0x7fb088074b60 2024-04-26T21:46:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.340+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 --> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2400002dd0 con 0x7f2408074be0 2024-04-26T21:46:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.342+0000 7f241ea00640 1 -- 172.21.15.106:0/3253873863 <== osd.0 v2:172.21.15.106:6800/2200991001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2400002dd0 con 0x7f2408074be0 2024-04-26T21:46:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.343+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f2408074be0 msgr2=0x7f2408077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.343+0000 7f242d600640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f2408074be0 0x7f2408077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.344+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 --> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe034002dd0 con 0x7fe038074be0 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.344+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f240806c9c0 msgr2=0x7f240806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.344+0000 7f242d600640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f240806c9c0 0x7f240806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f242815a590 tx=0x7f2418007e80 comp rx=0 tx=0).stop 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.344+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 msgr2=0x7f24281546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.344+0000 7f242d600640 1 --2- 172.21.15.106:0/3253873863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2428070580 0x7f24281546f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2420007ef0 tx=0x7f242000d4a0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7f2427400640 1 -- 172.21.15.106:0/3253873863 reap_dead start 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 shutdown_connections 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 >> 172.21.15.106:0/3253873863 conn(0x7f24280b3240 msgr2=0x7f24280b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7fe04b400640 1 -- 172.21.15.106:0/816600767 <== osd.1 v2:172.21.15.106:6804/1887774310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe034002dd0 con 0x7fe038074be0 2024-04-26T21:46:03.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 shutdown_connections 2024-04-26T21:46:03.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.345+0000 7f242d600640 1 -- 172.21.15.106:0/3253873863 wait complete. 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7fe038074be0 msgr2=0x7fe038077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7fe038074be0 0x7fe038077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe03806c9c0 msgr2=0x7fe03806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe03806c9c0 0x7fe03806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe0500ebf60 tx=0x7fe04c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 msgr2=0x7fe050050130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.346+0000 7fe05cc00640 1 --2- 172.21.15.106:0/816600767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05004bc80 0x7fe050050130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe04400f560 tx=0x7fe04400fa20 comp rx=0 tx=0).stop 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.347+0000 7fe057e00640 1 -- 172.21.15.106:0/816600767 reap_dead start 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.347+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 shutdown_connections 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.347+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 >> 172.21.15.106:0/816600767 conn(0x7fe0500559e0 msgr2=0x7fe05003a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.348+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 shutdown_connections 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.350+0000 7fe05cc00640 1 -- 172.21.15.106:0/816600767 wait complete. 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.353+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 --> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00880d1700 con 0x7f0068074b40 2024-04-26T21:46:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.354+0000 7f007be00640 1 -- 172.21.15.106:0/85379341 <== osd.2 v2:172.21.15.106:6824/323734454 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f00880d1700 con 0x7f0068074b40 2024-04-26T21:46:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.355+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f0068074b40 msgr2=0x7f0068076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.355+0000 7f008ee00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f0068074b40 0x7f0068076f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.356+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f006806c940 msgr2=0x7f006806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.356+0000 7f008ee00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f006806c940 0x7f006806ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f007c034000 tx=0x7f007c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.356+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 msgr2=0x7f008814f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.356+0000 7f008ee00640 1 --2- 172.21.15.106:0/85379341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00880d1b90 0x7f008814f1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f007400c480 tx=0x7f007400c940 comp rx=0 tx=0).stop 2024-04-26T21:46:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7f008ce00640 1 -- 172.21.15.106:0/85379341 reap_dead start 2024-04-26T21:46:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 --> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce400a75c0 con 0x7fce20074be0 2024-04-26T21:46:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 shutdown_connections 2024-04-26T21:46:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 >> 172.21.15.106:0/85379341 conn(0x7f008809e200 msgr2=0x7f008806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 shutdown_connections 2024-04-26T21:46:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.357+0000 7f008ee00640 1 -- 172.21.15.106:0/85379341 wait complete. 2024-04-26T21:46:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.358+0000 7fce3d600640 1 -- 172.21.15.106:0/3984117662 <== osd.3 v2:172.21.15.106:6816/56943883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce400a75c0 con 0x7fce20074be0 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7fce20074be0 msgr2=0x7fce20077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7fce20074be0 0x7fce20077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fce2006c9c0 msgr2=0x7fce2006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fce2006c9c0 0x7fce2006ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034000 tx=0x7fce2c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 msgr2=0x7fce401eef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.359+0000 7fce47a00640 1 --2- 172.21.15.106:0/3984117662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce400d6100 0x7fce401eef30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b9e0 tx=0x7fce3000bea0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.361+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 --> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9ec0578f0 con 0x7fa9d4074b60 2024-04-26T21:46:03.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.362+0000 7fce45a00640 1 -- 172.21.15.106:0/3984117662 reap_dead start 2024-04-26T21:46:03.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.362+0000 7fa9f1600640 1 -- 172.21.15.106:0/3512727915 <== osd.5 v2:172.21.15.204:6816/885497049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9ec0578f0 con 0x7fa9d4074b60 2024-04-26T21:46:03.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.362+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 shutdown_connections 2024-04-26T21:46:03.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.362+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 >> 172.21.15.106:0/3984117662 conn(0x7fce400bb0d0 msgr2=0x7fce40071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 shutdown_connections 2024-04-26T21:46:03.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa9d4074b60 msgr2=0x7fa9d4076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa9d4074b60 0x7fa9d4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9d406c940 msgr2=0x7fa9d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9d406c940 0x7fa9d406ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9ec00ab40 msgr2=0x7fa9ec04b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 --2- 172.21.15.106:0/3512727915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9ec00ab40 0x7fa9ec04b6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e850 tx=0x7fa9e800ed10 comp rx=0 tx=0).stop 2024-04-26T21:46:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.364+0000 7fce47a00640 1 -- 172.21.15.106:0/3984117662 wait complete. 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fa200640 1 -- 172.21.15.106:0/3512727915 reap_dead start 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 shutdown_connections 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.363+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 >> 172.21.15.106:0/3512727915 conn(0x7fa9ec056cd0 msgr2=0x7fa9ec036990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.364+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 shutdown_connections 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.364+0000 7fa9fc200640 1 -- 172.21.15.106:0/3512727915 wait complete. 2024-04-26T21:46:03.391 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.392 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:46:03.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.369+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 --> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3420069540 con 0x7f3400074a90 2024-04-26T21:46:03.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.369+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 --> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0a80ba5f0 con 0x7fb088074b60 2024-04-26T21:46:03.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.370+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 --> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5824002dd0 con 0x7f5828074a90 2024-04-26T21:46:03.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.370+0000 7f3416000640 1 -- 172.21.15.106:0/1604965177 <== osd.6 v2:172.21.15.204:6821/3330975067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3420069540 con 0x7f3400074a90 2024-04-26T21:46:03.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.370+0000 7fb09f400640 1 -- 172.21.15.106:0/3334406474 <== osd.7 v2:172.21.15.204:6808/2746439320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb0a000a720 con 0x7fb088074b60 2024-04-26T21:46:03.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.370+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f3400074a90 msgr2=0x7f3400076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fb088074b60 msgr2=0x7fb088076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fb088074b60 0x7fb088076f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.370+0000 7f3424a00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f3400074a90 0x7f3400076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f340006c8f0 msgr2=0x7f340006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f340006c8f0 0x7f340006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3410009bd0 tx=0x7f3410009240 comp rx=0 tx=0).stop 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342015ef60 msgr2=0x7f34200c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 --2- 172.21.15.106:0/1604965177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342015ef60 0x7f34200c94e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f341800c760 tx=0x7f341800cc20 comp rx=0 tx=0).stop 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f341e800640 1 -- 172.21.15.106:0/1604965177 reap_dead start 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 shutdown_connections 2024-04-26T21:46:03.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 >> 172.21.15.106:0/1604965177 conn(0x7f342009e3b0 msgr2=0x7f342006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb08806c940 msgr2=0x7fb08806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb08806c940 0x7fb08806ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098007890 comp rx=0 tx=0).stop 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 msgr2=0x7fb0a80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 --2- 172.21.15.106:0/3334406474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0a80d5d90 0x7fb0a80b8ce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb0940046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0a7e00640 1 -- 172.21.15.106:0/3334406474 reap_dead start 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 shutdown_connections 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 >> 172.21.15.106:0/3334406474 conn(0x7fb0a809e3b0 msgr2=0x7fb0a80b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.371+0000 7f583be00640 1 -- 172.21.15.106:0/2517515788 <== osd.4 v2:172.21.15.204:6800/3289538728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5824002dd0 con 0x7f5828074a90 2024-04-26T21:46:03.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.372+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 shutdown_connections 2024-04-26T21:46:03.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.372+0000 7fb0ae200640 1 -- 172.21.15.106:0/3334406474 wait complete. 2024-04-26T21:46:03.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.372+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 shutdown_connections 2024-04-26T21:46:03.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.372+0000 7f3424a00640 1 -- 172.21.15.106:0/1604965177 wait complete. 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7f5828074a90 msgr2=0x7f5828076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7f5828074a90 0x7f5828076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f582806c8f0 msgr2=0x7f582806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f582806c8f0 0x7f582806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5834009ba0 tx=0x7f5834008040 comp rx=0 tx=0).stop 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58400e1310 msgr2=0x7f584004d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.373+0000 7f584dc00640 1 --2- 172.21.15.106:0/2517515788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58400e1310 0x7f584004d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f583c031000 tx=0x7f583c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.399 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:46:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.374+0000 7f584d200640 1 -- 172.21.15.106:0/2517515788 reap_dead start 2024-04-26T21:46:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.375+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 shutdown_connections 2024-04-26T21:46:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.375+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 >> 172.21.15.106:0/2517515788 conn(0x7f5840056cd0 msgr2=0x7f5840055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.375+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 shutdown_connections 2024-04-26T21:46:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.375+0000 7f584dc00640 1 -- 172.21.15.106:0/2517515788 wait complete. 2024-04-26T21:46:03.401 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.401 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:46:03.402 INFO:teuthology.orchestra.run.smithi106.stdout:55834574853 2024-04-26T21:46:03.402 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:46:03.403 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.403 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:46:03.404 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:03.404 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:46:03.405 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.405 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:46:03.406 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:03.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:46:03.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.719+0000 7f6485e00640 1 Processor -- start 2024-04-26T21:46:03.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.721+0000 7ff2f9e00640 1 Processor -- start 2024-04-26T21:46:03.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.719+0000 7f6485e00640 1 -- start start 2024-04-26T21:46:03.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.721+0000 7f6485e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f647803c090 0x7f64780e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7f6485e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 0x7f64780540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7f6485e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6478058800 con 0x7f64780e3150 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7f647e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f647803c090 0x7f64780e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7f647e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f647803c090 0x7f64780e7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37502/0 (socket says 172.21.15.106:37502) 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7f647e000640 1 -- 172.21.15.106:0/3533953769 learned_addr learned my addr 172.21.15.106:0/3533953769 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.722+0000 7ff2f9e00640 1 -- start start 2024-04-26T21:46:03.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.723+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6478007b60 con 0x7f6478053ca0 2024-04-26T21:46:03.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.723+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6478058ae0 con 0x7f647803c090 2024-04-26T21:46:03.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.724+0000 7f647f400640 1 --2- 172.21.15.106:0/3533953769 >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 0x7f64780540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.724+0000 7f647e000640 1 -- 172.21.15.106:0/3533953769 >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 msgr2=0x7f64780540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.725+0000 7f647e000640 1 --2- 172.21.15.106:0/3533953769 >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 0x7f64780540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.725+0000 7f647e000640 1 -- 172.21.15.106:0/3533953769 >> v1:172.21.15.106:6789/0 conn(0x7f64780e3150 legacy=0x7f6478058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.725+0000 7f647e000640 1 -- 172.21.15.106:0/3533953769 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6478006350 con 0x7f647803c090 2024-04-26T21:46:03.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7f647f400640 1 --2- 172.21.15.106:0/3533953769 >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 0x7f64780540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff2ec0df410 0x7ff2ec04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff2ec0382b0 0x7ff2ec038760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff2ec006890 con 0x7ff2ec0df410 2024-04-26T21:46:03.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff2ec006ae0 con 0x7ff2ec0382b0 2024-04-26T21:46:03.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7f647e000640 1 --2- 172.21.15.106:0/3533953769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f647803c090 0x7f64780e7770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64780536f0 tx=0x7f646c02a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d336c8ca16ba27e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7f2441400640 1 Processor -- start 2024-04-26T21:46:03.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7f2441400640 1 -- start start 2024-04-26T21:46:03.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.726+0000 7ff2f9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec006dc0 con 0x7ff2ec00aa90 2024-04-26T21:46:03.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39554/0 (socket says 172.21.15.106:39554) 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7ff2f2000640 1 -- 172.21.15.106:0/3783904742 learned_addr learned my addr 172.21.15.106:0/3783904742 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.727+0000 7ff2f2a00640 1 --2- 172.21.15.106:0/3783904742 >> v2:172.21.15.106:3300/0 conn(0x7ff2ec0df410 0x7ff2ec04fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.729+0000 7fc494a00640 1 Processor -- start 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7ff2f2000640 1 -- 172.21.15.106:0/3783904742 >> v2:172.21.15.204:3300/0 conn(0x7ff2ec0382b0 msgr2=0x7ff2ec038760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7ff2f2000640 1 --2- 172.21.15.106:0/3783904742 >> v2:172.21.15.204:3300/0 conn(0x7ff2ec0382b0 0x7ff2ec038760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7ff2f2000640 1 -- 172.21.15.106:0/3783904742 >> v2:172.21.15.106:3300/0 conn(0x7ff2ec0df410 msgr2=0x7ff2ec04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7ff2f2000640 1 --2- 172.21.15.106:0/3783904742 >> v2:172.21.15.106:3300/0 conn(0x7ff2ec0df410 0x7ff2ec04fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.728+0000 7ff2f2000640 1 -- 172.21.15.106:0/3783904742 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec0070a0 con 0x7ff2ec00aa90 2024-04-26T21:46:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.729+0000 7f647d600640 1 -- 172.21.15.106:0/3533953769 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c037070 con 0x7f647803c090 2024-04-26T21:46:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.729+0000 7fc494a00640 1 -- start start 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f647d600640 1 -- 172.21.15.106:0/3533953769 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c032440 con 0x7f647803c090 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f2441400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f2441400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c0d3a00 con 0x7f243c0d3530 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f2441400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c0d3ce0 con 0x7f243c0b62a0 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f2441400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f243c0c88c0 con 0x7f243c0b6680 2024-04-26T21:46:03.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f6e2f200640 1 Processor -- start 2024-04-26T21:46:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.730+0000 7f647d600640 1 -- 172.21.15.106:0/3533953769 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c037070 con 0x7f647803c090 2024-04-26T21:46:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7ff2f2000640 1 --2- 172.21.15.106:0/3783904742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00dc50 tx=0x7ff2e402a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e248a307946fb0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f243be00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f243c0b62a0 0x7f243c0d2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:38328/0 (socket says 172.21.15.106:38328) 2024-04-26T21:46:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f243be00640 1 -- 172.21.15.106:0/1346852126 learned_addr learned my addr 172.21.15.106:0/1346852126 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f2440a00640 1 --2- 172.21.15.106:0/1346852126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f4a2b000640 1 Processor -- start 2024-04-26T21:46:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f6e2f200640 1 -- start start 2024-04-26T21:46:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7ff2f1600640 1 -- 172.21.15.106:0/3783904742 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4037070 con 0x7ff2ec00aa90 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7fc494a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 0x7fc49015f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc494a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc4900d34a0 0x7fc4900d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc48de00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc49015ef60 0x7fc4900b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58886/0 (socket says 172.21.15.106:58886) 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7ff2f1600640 1 -- 172.21.15.106:0/3783904742 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4032440 con 0x7ff2ec00aa90 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7ff2f1600640 1 -- 172.21.15.106:0/3783904742 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4032810 con 0x7ff2ec00aa90 2024-04-26T21:46:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.731+0000 7f4a2b000640 1 -- start start 2024-04-26T21:46:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc494a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4900b5d30 con 0x7fc49015ef60 2024-04-26T21:46:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc494a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc4900d3e40 con 0x7fc4900d34a0 2024-04-26T21:46:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc494a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4900b6010 con 0x7fc4900d5d90 2024-04-26T21:46:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f647803c090 msgr2=0x7f64780e7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7fc48de00640 1 -- 172.21.15.106:0/3855713433 learned_addr learned my addr 172.21.15.106:0/3855713433 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 --2- 172.21.15.106:0/3533953769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f647803c090 0x7f64780e7770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64780536f0 tx=0x7f646c02a280 comp rx=0 tx=0).stop 2024-04-26T21:46:03.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f2440a00640 1 -- 172.21.15.106:0/1346852126 >> v1:172.21.15.204:6789/0 conn(0x7f243c0b62a0 legacy=0x7f243c0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6e2f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6e300b2b30 0x7f6e300d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 0x7f6e300d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6e2f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e300d05b0 con 0x7f6e3015f3a0 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6e300d57c0 con 0x7f6e300b2b30 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6e300d0b90 con 0x7f6e300d1f60 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f2440a00640 1 -- 172.21.15.106:0/1346852126 >> v1:172.21.15.106:6789/0 conn(0x7f243c0d3530 legacy=0x7f243c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f2440a00640 1 -- 172.21.15.106:0/1346852126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0cfdd0 con 0x7f243c0b6680 2024-04-26T21:46:03.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 -- 172.21.15.106:0/3783904742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 msgr2=0x7ff2ec04ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/3783904742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00dc50 tx=0x7ff2e402a1f0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 0x7f6e300d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6eb8a00640 1 Processor -- start 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 -- 172.21.15.106:0/3783904742 shutdown_connections 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/3783904742 >> v2:172.21.15.204:3300/0 conn(0x7ff2ec0382b0 0x7ff2ec038760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/3783904742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec04ffc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6e300b2b30 0x7f6e300d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 0x7f6e300d3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39582/0 (socket says 172.21.15.106:39582) 2024-04-26T21:46:03.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7f6e2e800640 1 -- 172.21.15.106:0/3386381577 learned_addr learned my addr 172.21.15.106:0/3386381577 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/3783904742 >> v2:172.21.15.106:3300/0 conn(0x7ff2ec0df410 0x7ff2ec04fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7ff2f9e00640 1 -- 172.21.15.106:0/3783904742 >> 172.21.15.106:0/3783904742 conn(0x7ff2ec009fa0 msgr2=0x7ff2ec00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7ff2f9e00640 1 -- 172.21.15.106:0/3783904742 shutdown_connections 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 shutdown_connections 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 --2- 172.21.15.106:0/3533953769 >> v2:172.21.15.106:3300/0 conn(0x7f6478053ca0 0x7f64780540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 --2- 172.21.15.106:0/3533953769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f647803c090 0x7f64780e7770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.732+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 >> 172.21.15.106:0/3533953769 conn(0x7f6478058ed0 msgr2=0x7f64780592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c0548a0 0x7f4a1c054ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2b000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a1c03c360 0x7f4a1c03c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2b000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1c0e7e30 con 0x7f4a1c0e7a50 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2b000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c03b5a0 con 0x7f4a1c03c360 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c03b880 con 0x7f4a1c0548a0 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2a600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a1c03c360 0x7f4a1c03c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2a600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a1c03c360 0x7f4a1c03c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52082/0 (socket says 172.21.15.106:52082) 2024-04-26T21:46:03.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f4a2a600640 1 -- 172.21.15.106:0/379687628 learned_addr learned my addr 172.21.15.106:0/379687628 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7fc48e800640 1 --2- 172.21.15.106:0/3855713433 >> v2:172.21.15.204:3300/0 conn(0x7fc4900d34a0 0x7fc4900d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.733+0000 7fc48d400640 1 --2- 172.21.15.106:0/3855713433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 0x7fc49015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7fc48d400640 1 -- 172.21.15.106:0/3855713433 >> v2:172.21.15.204:3300/0 conn(0x7fc4900d34a0 msgr2=0x7fc4900d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7fc48d400640 1 --2- 172.21.15.106:0/3855713433 >> v2:172.21.15.204:3300/0 conn(0x7fc4900d34a0 0x7fc4900d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7fc48d400640 1 -- 172.21.15.106:0/3855713433 >> v1:172.21.15.106:6789/0 conn(0x7fc49015ef60 legacy=0x7fc4900b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7fc48d400640 1 -- 172.21.15.106:0/3855713433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4900b62f0 con 0x7fc4900d5d90 2024-04-26T21:46:03.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7f6eb8a00640 1 -- start start 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.734+0000 7ff2f9e00640 1 -- 172.21.15.106:0/3783904742 wait complete. 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 shutdown_connections 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6485e00640 1 -- 172.21.15.106:0/3533953769 wait complete. 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb8a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6eac03e840 0x7f6eac053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb8a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac058570 con 0x7f6eac0e30c0 2024-04-26T21:46:03.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb8a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac058850 con 0x7f6eac054130 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb8a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6eac0545a0 con 0x7f6eac03e840 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f4a2a600640 1 -- 172.21.15.106:0/379687628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c0548a0 msgr2=0x7f4a1c054ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f4a2a600640 1 --2- 172.21.15.106:0/379687628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c0548a0 0x7f4a1c054ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f4a2a600640 1 -- 172.21.15.106:0/379687628 >> v1:172.21.15.106:6789/0 conn(0x7f4a1c0e7a50 legacy=0x7f4a1c049b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f4a2a600640 1 -- 172.21.15.106:0/379687628 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c03bb60 con 0x7f4a1c03c360 2024-04-26T21:46:03.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb3400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6eac0e30c0 0x7f6eac057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58900/0 (socket says 172.21.15.106:58900) 2024-04-26T21:46:03.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f2440a00640 1 --2- 172.21.15.106:0/1346852126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c0b58b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f242c0097d0 tx=0x7f242c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50fe4bec8a5d518a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f6eb3400640 1 -- 172.21.15.106:0/1882556100 learned_addr learned my addr 172.21.15.106:0/1882556100 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.736+0000 7f4a2a600640 1 --2- 172.21.15.106:0/379687628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c03c360 0x7f4a1c03c7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a24051c10 tx=0x7f4a240a6860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eb650e22c3c144 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6e2d400640 1 -- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 msgr2=0x7f6e300d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6e2d400640 1 --2- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 0x7f6e300d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6e2d400640 1 -- 172.21.15.106:0/3386381577 >> v1:172.21.15.204:6789/0 conn(0x7f6e3015f3a0 legacy=0x7f6e300d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.735+0000 7f6e2d400640 1 -- 172.21.15.106:0/3386381577 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e300d5a10 con 0x7f6e300b2b30 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7fc48d400640 1 --2- 172.21.15.106:0/3855713433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 0x7fc49015f340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc488009800 tx=0x7fc488032a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c26a7af2b27e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f4a23e00640 1 -- 172.21.15.106:0/379687628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2406e6a0 con 0x7f4a1c03c360 2024-04-26T21:46:03.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/1882556100 >> v2:172.21.15.106:3300/0 conn(0x7f6eac03e840 0x7f6eac053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769060450 0 0) 0x7f6eac058570 con 0x7f6eac0e30c0 2024-04-26T21:46:03.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f4a23e00640 1 -- 172.21.15.106:0/379687628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2406e880 con 0x7f4a1c03c360 2024-04-26T21:46:03.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f4a23e00640 1 -- 172.21.15.106:0/379687628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2406ecf0 con 0x7f4a1c03c360 2024-04-26T21:46:03.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.737+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e9c003520 con 0x7f6eac0e30c0 2024-04-26T21:46:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6485e00640 1 Processor -- start 2024-04-26T21:46:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6485e00640 1 -- start start 2024-04-26T21:46:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504717067 0 0) 0x7f6e9c003520 con 0x7f6eac0e30c0 2024-04-26T21:46:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 >> v2:172.21.15.106:3300/0 conn(0x7f6eac03e840 msgr2=0x7f6eac053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 --2- 172.21.15.106:0/1882556100 >> v2:172.21.15.106:3300/0 conn(0x7f6eac03e840 0x7f6eac053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7ff2f9e00640 1 Processor -- start 2024-04-26T21:46:03.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 >> v1:172.21.15.204:6789/0 conn(0x7f6eac054130 legacy=0x7f6eac0068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6e2d400640 1 --2- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300b2b30 0x7f6e300d2ac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0097a0 tx=0x7f6e1c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc80eff11832ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7ff2f9e00640 1 -- start start 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f4a2b000640 1 -- 172.21.15.106:0/379687628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c03c360 msgr2=0x7f4a1c03c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f4a2b000640 1 --2- 172.21.15.106:0/379687628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c03c360 0x7f4a1c03c7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a24051c10 tx=0x7f4a240a6860 comp rx=0 tx=0).stop 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 -- 172.21.15.106:0/379687628 shutdown_connections 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 --2- 172.21.15.106:0/379687628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c03c360 0x7f4a1c03c7f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 --2- 172.21.15.106:0/379687628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c0548a0 0x7f4a1c054ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac058b30 con 0x7f6eac0e30c0 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f6eb406a4b0 con 0x7f6eac0e30c0 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 >> v1:172.21.15.106:6789/0 conn(0x7f6eac0e30c0 legacy=0x7f6eac057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 0x7f6e9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c004350 0x7f6e9c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9c004fd0 0x7f6e9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0545a0 con 0x7f6e9c0039b0 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c00bf60 con 0x7f6e9c004fd0 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9c00c160 con 0x7f6e9c004350 2024-04-26T21:46:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.738+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/1882556100 >> v2:172.21.15.106:3300/0 conn(0x7f6eac03e840 0x7f6eac053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 -- 172.21.15.106:0/379687628 >> 172.21.15.106:0/379687628 conn(0x7f4a1c056cd0 msgr2=0x7f4a1c0570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6eb3e00640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9c004fd0 0x7f6e9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6eb3400640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 0x7f6e9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 -- 172.21.15.106:0/379687628 shutdown_connections 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6478053ca0 0x7f647804e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 0x7f647804ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f64780e3530 con 0x7f64780e3150 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6478054630 con 0x7f6478053ca0 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6485e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6478006b50 con 0x7f647803c090 2024-04-26T21:46:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f4a2b000640 1 -- 172.21.15.106:0/379687628 wait complete. 2024-04-26T21:46:03.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6e2ca00640 1 -- 172.21.15.106:0/3386381577 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c037070 con 0x7f6e300b2b30 2024-04-26T21:46:03.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6e2ca00640 1 -- 172.21.15.106:0/3386381577 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c02ae40 con 0x7f6e300b2b30 2024-04-26T21:46:03.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c004350 0x7f6e9c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.739+0000 7f6e2ca00640 1 -- 172.21.15.106:0/3386381577 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c032660 con 0x7f6e300b2b30 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f647f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 0x7f647804ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f6eb3400640 1 -- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c004350 msgr2=0x7f6e9c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f6eb3400640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c004350 0x7f6e9c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f6eb3400640 1 -- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9c004fd0 msgr2=0x7f6e9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f6eb3400640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9c004fd0 0x7f6e9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f6eb3400640 1 -- 172.21.15.106:0/1882556100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac058b30 con 0x7f6e9c0039b0 2024-04-26T21:46:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f647ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f647ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39602/0 (socket says 172.21.15.106:39602) 2024-04-26T21:46:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f647ea00640 1 -- 172.21.15.106:0/3584413313 learned_addr learned my addr 172.21.15.106:0/3584413313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f647e000640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6478053ca0 0x7f647804e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f243aa00640 1 -- 172.21.15.106:0/1346852126 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c037070 con 0x7f243c0b6680 2024-04-26T21:46:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f243aa00640 1 -- 172.21.15.106:0/1346852126 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c032440 con 0x7f243c0b6680 2024-04-26T21:46:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7ff2f9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec0dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7ff2f9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7ff2f9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7f243aa00640 1 -- 172.21.15.106:0/1346852126 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c032810 con 0x7f243c0b6680 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.740+0000 7fc48ca00640 1 -- 172.21.15.106:0/3855713433 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488036820 con 0x7fc4900d5d90 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7fc48ca00640 1 -- 172.21.15.106:0/3855713433 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4880369c0 con 0x7fc4900d5d90 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7fc48ca00640 1 -- 172.21.15.106:0/3855713433 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488036d90 con 0x7fc4900d5d90 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 -- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300b2b30 msgr2=0x7f6e300d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 --2- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300b2b30 0x7f6e300d2ac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0097a0 tx=0x7f6e1c02a130 comp rx=0 tx=0).stop 2024-04-26T21:46:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6eb3e00640 1 -- 172.21.15.106:0/1882556100 reap_dead start 2024-04-26T21:46:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 -- 172.21.15.106:0/3386381577 shutdown_connections 2024-04-26T21:46:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 --2- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300d1f60 0x7f6e300d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 --2- 172.21.15.106:0/3386381577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300b2b30 0x7f6e300d2ac0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 -- 172.21.15.106:0/3386381577 >> 172.21.15.106:0/3386381577 conn(0x7f6e3009e1c0 msgr2=0x7f6e300d4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f2441400640 1 -- 172.21.15.106:0/1346852126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 msgr2=0x7f243c0b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f647f400640 1 -- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 msgr2=0x7f64780de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647f400640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f6e2f200640 1 -- 172.21.15.106:0/3386381577 shutdown_connections 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647f400640 1 -- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6478053ca0 msgr2=0x7f647804e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647f400640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6478053ca0 0x7f647804e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647ea00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 -- 172.21.15.106:0/3855713433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 msgr2=0x7fc49015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 --2- 172.21.15.106:0/3855713433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 0x7fc49015f340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc488009800 tx=0x7fc488032a60 comp rx=0 tx=0).stop 2024-04-26T21:46:03.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f6e2f200640 1 -- 172.21.15.106:0/3386381577 wait complete. 2024-04-26T21:46:03.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647f400640 1 -- 172.21.15.106:0/3584413313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64780e42a0 con 0x7f64780e3150 2024-04-26T21:46:03.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7f2441400640 1 --2- 172.21.15.106:0/1346852126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c0b58b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f242c0097d0 tx=0x7f242c02a210 comp rx=0 tx=0).stop 2024-04-26T21:46:03.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f6eb3400640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 0x7f6e9c003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6eb406bba0 tx=0x7f6e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff54c4b4af2d535d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f2441400640 1 -- 172.21.15.106:0/1346852126 shutdown_connections 2024-04-26T21:46:03.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f2441400640 1 --2- 172.21.15.106:0/1346852126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c0b58b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f2441400640 1 -- 172.21.15.106:0/1346852126 >> 172.21.15.106:0/1346852126 conn(0x7f243c075d10 msgr2=0x7f243c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 -- 172.21.15.106:0/3855713433 shutdown_connections 2024-04-26T21:46:03.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 --2- 172.21.15.106:0/3855713433 >> v2:172.21.15.204:3300/0 conn(0x7fc4900d34a0 0x7fc4900d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 --2- 172.21.15.106:0/3855713433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4900d5d90 0x7fc49015f340 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc488009800 tx=0x7fc488032a60 comp rx=0 tx=0).stop 2024-04-26T21:46:03.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7f647f400640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 0x7f647804ffa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6480000b60 tx=0x7f6480076830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7fc494a00640 1 -- 172.21.15.106:0/3855713433 >> 172.21.15.106:0/3855713433 conn(0x7fc49009e3b0 msgr2=0x7fc4900d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.741+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37546/0 (socket says 172.21.15.106:37546) 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52120/0 (socket says 172.21.15.106:52120) 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f9e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec052cb0 con 0x7ff2ec0df410 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec052b30 con 0x7ff2ec0382b0 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec038ca0 con 0x7ff2ec00aa90 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f2000640 1 -- 172.21.15.106:0/1480435507 learned_addr learned my addr 172.21.15.106:0/1480435507 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.742+0000 7ff2f2a00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec0dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7fc494a00640 1 -- 172.21.15.106:0/3855713433 shutdown_connections 2024-04-26T21:46:03.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7fc494a00640 1 -- 172.21.15.106:0/3855713433 wait complete. 2024-04-26T21:46:03.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7fdc9be00640 1 Processor -- start 2024-04-26T21:46:03.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7fdc9be00640 1 -- start start 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7f2441400640 1 -- 172.21.15.106:0/1346852126 shutdown_connections 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7ff2f2000640 1 -- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 msgr2=0x7ff2ec0dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7ff2f2000640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec0dcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7ff2f2000640 1 -- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 msgr2=0x7ff2ec180b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7ff2f2000640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.743+0000 7ff2f2000640 1 -- 172.21.15.106:0/1480435507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec04bb80 con 0x7ff2ec0382b0 2024-04-26T21:46:03.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7f6e2f200640 1 Processor -- start 2024-04-26T21:46:03.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7f6e2f200640 1 -- start start 2024-04-26T21:46:03.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7f4a2b000640 1 Processor -- start 2024-04-26T21:46:03.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7ff2f2000640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c00 tx=0x7ff2e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7ff2f3400640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T21:46:03.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.744+0000 7f2441400640 1 -- 172.21.15.106:0/1346852126 wait complete. 2024-04-26T21:46:03.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f4a2b000640 1 -- start start 2024-04-26T21:46:03.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480076960 con 0x7f64780e3150 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647804ec40 con 0x7f64780e3150 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7fdc9be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdc9403e840 0x7fdc94053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7fdc9be00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc94058570 con 0x7fdc940e30c0 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7fdc9be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc94058850 con 0x7fdc94054130 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300d1f60 0x7f6e30154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e3015f3a0 0x7f6e300b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e300b3cf0 con 0x7f6e3015f3a0 2024-04-26T21:46:03.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6e300d5cf0 con 0x7f6e300d1f60 2024-04-26T21:46:03.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.745+0000 7fdc9be00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940545a0 con 0x7fdc9403e840 2024-04-26T21:46:03.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6e300b3e70 con 0x7f6e300b2b30 2024-04-26T21:46:03.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7fdc9aa00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdc940e30c0 0x7fdc94057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58916/0 (socket says 172.21.15.106:58916) 2024-04-26T21:46:03.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7fdc9aa00640 1 -- 172.21.15.106:0/21684744 learned_addr learned my addr 172.21.15.106:0/21684744 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:03.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40b4070 con 0x7f6e9c0039b0 2024-04-26T21:46:03.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a1c0548a0 0x7f4a1c008dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c0e7a50 0x7f4a1c0dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c04a270 con 0x7f4a1c0e7a50 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c04a5b0 con 0x7f4a1c0548a0 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f4a2b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c039cb0 con 0x7f4a1c03c360 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7fc494a00640 1 Processor -- start 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6480074830 con 0x7f64780e3150 2024-04-26T21:46:03.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64780e4520 con 0x7f64780e3150 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39608/0 (socket says 172.21.15.106:39608) 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2de00640 1 -- 172.21.15.106:0/639077728 learned_addr learned my addr 172.21.15.106:0/639077728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7f6e2d400640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300d1f60 0x7f6e30154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2e800640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e3015f3a0 0x7f6e300b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.746+0000 7fdc9a000640 1 --2- 172.21.15.106:0/21684744 >> v2:172.21.15.106:3300/0 conn(0x7fdc9403e840 0x7fdc94053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648006f660 con 0x7f64780e3150 2024-04-26T21:46:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7fc494a00640 1 -- start start 2024-04-26T21:46:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7fdc9a000640 1 -- 172.21.15.106:0/21684744 >> v1:172.21.15.204:6789/0 conn(0x7fdc94054130 legacy=0x7fdc940068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7fdc9a000640 1 -- 172.21.15.106:0/21684744 >> v1:172.21.15.106:6789/0 conn(0x7fdc940e30c0 legacy=0x7fdc94057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7fdc9a000640 1 -- 172.21.15.106:0/21684744 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc94058b30 con 0x7fdc9403e840 2024-04-26T21:46:03.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2de00640 1 -- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300d1f60 msgr2=0x7f6e30154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2de00640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300d1f60 0x7f6e30154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2de00640 1 -- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e3015f3a0 msgr2=0x7f6e300b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2de00640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e3015f3a0 0x7f6e300b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6e2de00640 1 -- 172.21.15.106:0/639077728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e300b90e0 con 0x7f6e300b2b30 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb40a7b00 con 0x7f6e9c0039b0 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.747+0000 7f6eb2000640 1 -- 172.21.15.106:0/1882556100 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4069610 con 0x7f6e9c0039b0 2024-04-26T21:46:03.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f4a29c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f4a29c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39610/0 (socket says 172.21.15.106:39610) 2024-04-26T21:46:03.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f4a29c00640 1 -- 172.21.15.106:0/2837828818 learned_addr learned my addr 172.21.15.106:0/2837828818 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f6e2de00640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000b3f0 tx=0x7f6e2000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f6eb8a00640 1 -- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 msgr2=0x7f6e9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 0x7f6e9c003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6eb406bba0 tx=0x7f6e9c005830 comp rx=0 tx=0).stop 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4900d34a0 0x7fc4900c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc49015ef60 0x7fc4900c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4900d01d0 con 0x7fc49015ef60 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4900b68b0 con 0x7fc4900d34a0 2024-04-26T21:46:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.748+0000 7fc494a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4900d0350 con 0x7fc490154f00 2024-04-26T21:46:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20015020 con 0x7f6e300b2b30 2024-04-26T21:46:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e20011830 con 0x7f6e300b2b30 2024-04-26T21:46:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29200640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a1c0548a0 0x7f4a1c008dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f2441400640 1 Processor -- start 2024-04-26T21:46:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6eb8a00640 1 -- 172.21.15.106:0/1882556100 shutdown_connections 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/1882556100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e9c0039b0 0x7f6e9c003dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6eb8a00640 1 -- 172.21.15.106:0/1882556100 >> 172.21.15.106:0/1882556100 conn(0x7f6eac0559e0 msgr2=0x7f6eac055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29c00640 1 -- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a1c0548a0 msgr2=0x7f4a1c008dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29c00640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a1c0548a0 0x7f4a1c008dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29c00640 1 -- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c0e7a50 msgr2=0x7f4a1c0dfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29c00640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c0e7a50 0x7f4a1c0dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f4a29c00640 1 -- 172.21.15.106:0/2837828818 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c04cad0 con 0x7f4a1c03c360 2024-04-26T21:46:03.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f2441400640 1 -- start start 2024-04-26T21:46:03.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7fc48e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7fc48e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39616/0 (socket says 172.21.15.106:39616) 2024-04-26T21:46:03.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7fc48e800640 1 -- 172.21.15.106:0/1121038266 learned_addr learned my addr 172.21.15.106:0/1121038266 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20011c20 con 0x7f6e300b2b30 2024-04-26T21:46:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e30160ae0 con 0x7f6e300b2b30 2024-04-26T21:46:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.749+0000 7f6eb8a00640 1 -- 172.21.15.106:0/1882556100 shutdown_connections 2024-04-26T21:46:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6458005290 con 0x7f64780e3150 2024-04-26T21:46:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e300b9660 con 0x7f6e300b2b30 2024-04-26T21:46:03.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48de00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4900d34a0 0x7fc4900c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f648006f800 con 0x7f64780e3150 2024-04-26T21:46:03.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fdc9a000640 1 --2- 172.21.15.106:0/21684744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc94053bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc8802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9df3c1a388e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48e800640 1 -- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4900d34a0 msgr2=0x7fc4900c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48e800640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4900d34a0 0x7fc4900c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48e800640 1 -- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc49015ef60 msgr2=0x7fc4900c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48e800640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc49015ef60 0x7fc4900c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7fc48e800640 1 -- 172.21.15.106:0/1121038266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4900c9990 con 0x7fc490154f00 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7f4a29c00640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c3f0 tx=0x7f4a1800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a180052d0 con 0x7f4a1c03c360 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a18005490 con 0x7f4a1c03c360 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18007260 con 0x7f4a1c03c360 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7fc48e800640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc48000efc0 tx=0x7fc48000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e30069ad0 con 0x7f6e300b2b30 2024-04-26T21:46:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.751+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480018020 con 0x7fc490154f00 2024-04-26T21:46:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4900c96b0 con 0x7fc490154f00 2024-04-26T21:46:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7f6477400640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 0x7f646006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480014480 con 0x7fc490154f00 2024-04-26T21:46:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4901f22c0 con 0x7fc490154f00 2024-04-26T21:46:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480014870 con 0x7fc490154f00 2024-04-26T21:46:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.750+0000 7f6eb8a00640 1 -- 172.21.15.106:0/1882556100 wait complete. 2024-04-26T21:46:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6480108040 con 0x7f64780e3150 2024-04-26T21:46:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7f6eb8a00640 1 Processor -- start 2024-04-26T21:46:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7f647ea00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 0x7f646006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.752+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4037040 con 0x7ff2ec0382b0 2024-04-26T21:46:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4042070 con 0x7ff2ec0382b0 2024-04-26T21:46:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4030940 con 0x7ff2ec0382b0 2024-04-26T21:46:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c04cdb0 con 0x7f4a1c03c360 2024-04-26T21:46:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c04d370 con 0x7f4a1c03c360 2024-04-26T21:46:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7f647ea00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 0x7f646006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6470007fd0 tx=0x7f6470009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec0c7910 con 0x7ff2ec0382b0 2024-04-26T21:46:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.753+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc468005290 con 0x7fc490154f00 2024-04-26T21:46:03.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f6eb8a00640 1 -- start start 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0d3530 0x7f243c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0c8da0 con 0x7f243c0d3530 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0d3fc0 con 0x7f243c0b62a0 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f2441400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f243c0c8f20 con 0x7f243c0b6680 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f243be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f243be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37580/0 (socket says 172.21.15.106:37580) 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f243be00640 1 -- 172.21.15.106:0/79072458 learned_addr learned my addr 172.21.15.106:0/79072458 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec04be20 con 0x7ff2ec0382b0 2024-04-26T21:46:03.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f243b400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e20018020 con 0x7f6e300b2b30 2024-04-26T21:46:03.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6e26000640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 0x7f6e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.754+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c0057e0 con 0x7f4a1c03c360 2024-04-26T21:46:03.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e2008b810 con 0x7f6e300b2b30 2024-04-26T21:46:03.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6e2d400640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 0x7f6e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c0039b0 0x7f6eac04ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eac0db030 0x7f6eac0db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0e34a0 con 0x7f6eac0d96c0 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac03bfc0 con 0x7f6eac0db030 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f6eb8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6eac053bf0 con 0x7f6e9c0039b0 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f6e2d400640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 0x7f6e1006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c034000 tx=0x7f6e1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f243be00640 1 -- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 msgr2=0x7f243c15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f243be00640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f243be00640 1 -- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0d3530 msgr2=0x7f243c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f243be00640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0d3530 0x7f243c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f6eb2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f6eb2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52172/0 (socket says 172.21.15.106:52172) 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f6eb2a00640 1 -- 172.21.15.106:0/4154676425 learned_addr learned my addr 172.21.15.106:0/4154676425 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.755+0000 7f243be00640 1 -- 172.21.15.106:0/79072458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0cf260 con 0x7f243c0b62a0 2024-04-26T21:46:03.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f243b400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:03.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f243be00640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f242800c4a0 tx=0x7f242800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.756+0000 7f6eb3400640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c0039b0 0x7f6eac04ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb3e00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eac0db030 0x7f6eac0db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec0089c0 con 0x7ff2ec0382b0 2024-04-26T21:46:03.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428017070 con 0x7f243c0b62a0 2024-04-26T21:46:03.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c0cdf20 con 0x7f243c0b62a0 2024-04-26T21:46:03.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c0ce780 con 0x7f243c0b62a0 2024-04-26T21:46:03.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 -- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c0039b0 msgr2=0x7f6eac04ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c0039b0 0x7f6eac04ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 -- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eac0db030 msgr2=0x7f6eac0db4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eac0db030 0x7f6eac0db4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 -- 172.21.15.106:0/4154676425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac0dd380 con 0x7f6eac0d96c0 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2428005ce0 con 0x7f243c0b62a0 2024-04-26T21:46:03.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f6eb2a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4011900 tx=0x7f6ea4011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428007270 con 0x7f243c0b62a0 2024-04-26T21:46:03.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.757+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2e4030ae0 con 0x7ff2ec0382b0 2024-04-26T21:46:03.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.758+0000 7ff2eb400640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 0x7ff2d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.759+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e40ac1c0 con 0x7ff2ec0382b0 2024-04-26T21:46:03.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.759+0000 7ff2f2a00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 0x7ff2d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.759+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2414005290 con 0x7f243c0b62a0 2024-04-26T21:46:03.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.759+0000 7ff2f2a00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 0x7ff2d406ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009aa0 tx=0x7ff2e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.760+0000 7fdc99600640 1 -- 172.21.15.106:0/21684744 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88037070 con 0x7fdc9403e840 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.760+0000 7fdc99600640 1 -- 172.21.15.106:0/21684744 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc88032640 con 0x7fdc9403e840 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.760+0000 7fdc99600640 1 -- 172.21.15.106:0/21684744 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88032a10 con 0x7fdc9403e840 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64780e42a0 con 0x7f64780e3150 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400fbd0 con 0x7f6eac0d96c0 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea400fd70 con 0x7f6eac0d96c0 2024-04-26T21:46:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4015660 con 0x7f6eac0d96c0 2024-04-26T21:46:03.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 -- 172.21.15.106:0/21684744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 msgr2=0x7fdc94053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 --2- 172.21.15.106:0/21684744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc94053bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc8802a2c0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac0dd660 con 0x7f6eac0d96c0 2024-04-26T21:46:03.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac0ddc20 con 0x7f6eac0d96c0 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 -- 172.21.15.106:0/21684744 shutdown_connections 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 --2- 172.21.15.106:0/21684744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc94053bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 -- 172.21.15.106:0/21684744 >> 172.21.15.106:0/21684744 conn(0x7fdc940559e0 msgr2=0x7fdc94055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 -- 172.21.15.106:0/21684744 shutdown_connections 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.761+0000 7fdc9be00640 1 -- 172.21.15.106:0/21684744 wait complete. 2024-04-26T21:46:03.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.762+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac0057e0 con 0x7f6eac0d96c0 2024-04-26T21:46:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.763+0000 7fdc9be00640 1 Processor -- start 2024-04-26T21:46:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.763+0000 7fdc9be00640 1 -- start start 2024-04-26T21:46:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.763+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea4015800 con 0x7f6eac0d96c0 2024-04-26T21:46:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc94054130 0x7fdc940c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940e30c0 0x7fdc9403d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc940e34a0 con 0x7fdc9403e840 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9403bfc0 con 0x7fdc94054130 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdc94006fe0 con 0x7fdc940e30c0 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7f6ea3400640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 0x7f6e9806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea408c560 con 0x7f6eac0d96c0 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7f6eb3400640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 0x7f6e9806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52184/0 (socket says 172.21.15.106:52184) 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.765+0000 7fdc9aa00640 1 -- 172.21.15.106:0/3002960181 learned_addr learned my addr 172.21.15.106:0/3002960181 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 -- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940e30c0 msgr2=0x7fdc9403d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940e30c0 0x7fdc9403d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 -- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc94054130 msgr2=0x7fdc940c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc94054130 0x7fdc940c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 -- 172.21.15.106:0/3002960181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc94052a70 con 0x7fdc9403e840 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7fdc9aa00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00c3c0 tx=0x7fdc8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.766+0000 7f6eb3400640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 0x7f6e9806ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00bf30 tx=0x7f6eb40ab000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c017070 con 0x7fdc9403e840 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c005ce0 con 0x7fdc9403e840 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0072f0 con 0x7fdc9403e840 2024-04-26T21:46:03.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc94052d10 con 0x7fdc9403e840 2024-04-26T21:46:03.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ea4019030 con 0x7f6eac0d96c0 2024-04-26T21:46:03.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.769+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc940dca60 con 0x7fdc9403e840 2024-04-26T21:46:03.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.771+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc940057e0 con 0x7fdc9403e840 2024-04-26T21:46:03.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.771+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e2001f030 con 0x7f6e300b2b30 2024-04-26T21:46:03.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.771+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc8c01a070 con 0x7fdc9403e840 2024-04-26T21:46:03.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.773+0000 7fdc93400640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 0x7fdc7c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.773+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc8c08aec0 con 0x7fdc9403e840 2024-04-26T21:46:03.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.773+0000 7fdc9a000640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 0x7fdc7c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.775+0000 7fdc9a000640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 0x7fdc7c06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc88034000 tx=0x7fdc8802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.775+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc48001b020 con 0x7fc490154f00 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.774+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a18007400 con 0x7f4a1c03c360 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.775+0000 7f4a21600640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 0x7f4a0406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.775+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a1808b390 con 0x7f4a1c03c360 2024-04-26T21:46:03.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.776+0000 7f4a29200640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 0x7f4a0406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.776+0000 7fc486000640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.776+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc48008ca20 con 0x7fc490154f00 2024-04-26T21:46:03.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.777+0000 7f4a29200640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 0x7f4a0406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c049a10 tx=0x7f4a10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.777+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2e407a6d0 con 0x7ff2ec0382b0 2024-04-26T21:46:03.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.777+0000 7fc48de00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.778+0000 7fc48de00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc478007fd0 tx=0x7fc478007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.779+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f242801a070 con 0x7f243c0b62a0 2024-04-26T21:46:03.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.779+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc8c059360 con 0x7fdc9403e840 2024-04-26T21:46:03.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.780+0000 7f2433e00640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:03.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.780+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f242808b2c0 con 0x7f243c0b62a0 2024-04-26T21:46:03.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.780+0000 7f243b400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:03.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.781+0000 7f243b400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 0x7f241c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f243c0761a0 tx=0x7f2434009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:03.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.785+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc480022030 con 0x7fc490154f00 2024-04-26T21:46:03.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.791+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2428059000 con 0x7f243c0b62a0 2024-04-26T21:46:03.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:03.796+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a18059930 con 0x7f4a1c03c360 2024-04-26T21:46:04.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.195+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6458005580 con 0x7f64780e3150 2024-04-26T21:46:04.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.196+0000 7f6477400640 1 -- 172.21.15.106:0/3584413313 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64800d5e10 con 0x7f64780e3150 2024-04-26T21:46:04.197 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.200+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 msgr2=0x7f646006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.200+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 0x7f646006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6470007fd0 tx=0x7f6470009040 comp rx=0 tx=0).stop 2024-04-26T21:46:04.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.201+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 msgr2=0x7f647804ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.201+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 0x7f647804ffa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6480000b60 tx=0x7f6480076830 comp rx=0 tx=0).stop 2024-04-26T21:46:04.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 shutdown_connections 2024-04-26T21:46:04.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780e3150 0x7f647804ffa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6478053ca0 0x7f647804e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f646006c8f0 0x7f646006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 --2- 172.21.15.106:0/3584413313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f647803c090 0x7f64780de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 >> 172.21.15.106:0/3584413313 conn(0x7f6478058ed0 msgr2=0x7f64780e6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 shutdown_connections 2024-04-26T21:46:04.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.202+0000 7f6485e00640 1 -- 172.21.15.106:0/3584413313 wait complete. 2024-04-26T21:46:04.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.203+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdc94052ee0 con 0x7fdc9403e840 2024-04-26T21:46:04.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.204+0000 7fdc93400640 1 -- 172.21.15.106:0/3002960181 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc8c058c60 con 0x7fdc9403e840 2024-04-26T21:46:04.205 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 msgr2=0x7fdc7c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6e30068120 con 0x7f6e300b2b30 2024-04-26T21:46:04.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 0x7fdc7c06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc88034000 tx=0x7fdc8802c000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 msgr2=0x7fdc9404ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00c3c0 tx=0x7fdc8c00c880 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 shutdown_connections 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdc7c06c940 0x7fdc7c06ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940e30c0 0x7fdc9403d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc94054130 0x7fdc940c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 --2- 172.21.15.106:0/3002960181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc9403e840 0x7fdc9404ffa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.208+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 >> 172.21.15.106:0/3002960181 conn(0x7fdc940559e0 msgr2=0x7fdc940e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.209+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 shutdown_connections 2024-04-26T21:46:04.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.209+0000 7f6e26000640 1 -- 172.21.15.106:0/639077728 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e20059cb0 con 0x7f6e300b2b30 2024-04-26T21:46:04.210 INFO:teuthology.orchestra.run.smithi106.stdout:55834574853 2024-04-26T21:46:04.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.209+0000 7fdc9be00640 1 -- 172.21.15.106:0/3002960181 wait complete. 2024-04-26T21:46:04.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.209+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6eac057520 con 0x7f6eac0d96c0 2024-04-26T21:46:04.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.210+0000 7f6ea3400640 1 -- 172.21.15.106:0/4154676425 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ea405aa00 con 0x7f6eac0d96c0 2024-04-26T21:46:04.212 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 msgr2=0x7f6e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 0x7f6e1006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c034000 tx=0x7f6e1c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 msgr2=0x7f6e300cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000b3f0 tx=0x7f6e2000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 shutdown_connections 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e3015f3a0 0x7f6e300b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e300d1f60 0x7f6e30154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e1006c8f0 0x7f6e1006edb0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c034000 tx=0x7f6e1c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 --2- 172.21.15.106:0/639077728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e300b2b30 0x7f6e300cd010 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000b3f0 tx=0x7f6e2000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.213+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 >> 172.21.15.106:0/639077728 conn(0x7f6e3009e1c0 msgr2=0x7f6e300d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 shutdown_connections 2024-04-26T21:46:04.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6e2f200640 1 -- 172.21.15.106:0/639077728 wait complete. 2024-04-26T21:46:04.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 msgr2=0x7f6e9806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 0x7f6e9806ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00bf30 tx=0x7f6eb40ab000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 msgr2=0x7f6eac04b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.214+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4011900 tx=0x7f6ea4011dc0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 shutdown_connections 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eac0db030 0x7f6eac0db4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6eac0d96c0 0x7f6eac04b9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6e9806c970 0x7f6e9806ee30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 --2- 172.21.15.106:0/4154676425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e9c0039b0 0x7f6eac04ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 >> 172.21.15.106:0/4154676425 conn(0x7f6eac0559e0 msgr2=0x7f6eac039030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 shutdown_connections 2024-04-26T21:46:04.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.215+0000 7f6eb8a00640 1 -- 172.21.15.106:0/4154676425 wait complete. 2024-04-26T21:46:04.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.217+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff2ec0ee030 con 0x7ff2ec0382b0 2024-04-26T21:46:04.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.218+0000 7ff2eb400640 1 -- 172.21.15.106:0/1480435507 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2ec0ee030 con 0x7ff2ec0382b0 2024-04-26T21:46:04.223 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 msgr2=0x7ff2d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 0x7ff2d406ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009aa0 tx=0x7ff2e0009190 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 msgr2=0x7ff2ec0c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c00 tx=0x7ff2e40046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 shutdown_connections 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec0df410 0x7ff2ec180b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec0382b0 0x7ff2ec0c6eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff2d406c940 0x7ff2d406ee00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 --2- 172.21.15.106:0/1480435507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2ec00aa90 0x7ff2ec0dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.224+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 >> 172.21.15.106:0/1480435507 conn(0x7ff2ec009fa0 msgr2=0x7ff2ec039af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.225+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 shutdown_connections 2024-04-26T21:46:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.225+0000 7ff2f9e00640 1 -- 172.21.15.106:0/1480435507 wait complete. 2024-04-26T21:46:04.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-26T21:46:04.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.230+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc468005850 con 0x7fc490154f00 2024-04-26T21:46:04.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.231+0000 7fc486000640 1 -- 172.21.15.106:0/1121038266 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc480010040 con 0x7fc490154f00 2024-04-26T21:46:04.234 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.234+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2414005850 con 0x7f243c0b62a0 2024-04-26T21:46:04.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7f2433e00640 1 -- 172.21.15.106:0/79072458 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f242805d300 con 0x7f243c0b62a0 2024-04-26T21:46:04.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 msgr2=0x7fc46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc478007fd0 tx=0x7fc478007e80 comp rx=0 tx=0).stop 2024-04-26T21:46:04.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 msgr2=0x7fc49015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc48000efc0 tx=0x7fc48000c490 comp rx=0 tx=0).stop 2024-04-26T21:46:04.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 shutdown_connections 2024-04-26T21:46:04.236 INFO:teuthology.orchestra.run.smithi106.stdout:55834574852 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc478007fd0 tx=0x7fc478007e80 comp rx=0 tx=0).stop 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc490154f00 0x7fc49015dfb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc49015ef60 0x7fc4900c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 --2- 172.21.15.106:0/1121038266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4900d34a0 0x7fc4900c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.235+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 >> 172.21.15.106:0/1121038266 conn(0x7fc49009e3b0 msgr2=0x7fc4900b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.236+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4a1c00b240 con 0x7f4a1c03c360 2024-04-26T21:46:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.237+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 shutdown_connections 2024-04-26T21:46:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.237+0000 7fc494a00640 1 -- 172.21.15.106:0/1121038266 wait complete. 2024-04-26T21:46:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.237+0000 7f4a21600640 1 -- 172.21.15.106:0/2837828818 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4a18059230 con 0x7f4a1c03c360 2024-04-26T21:46:04.238 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:04.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-26T21:46:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.238+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 msgr2=0x7f241c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.238+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 0x7f241c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f243c0761a0 tx=0x7f2434009040 comp rx=0 tx=0).stop 2024-04-26T21:46:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.238+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 msgr2=0x7f243c1f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.238+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f242800c4a0 tx=0x7f242800c960 comp rx=0 tx=0).stop 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 shutdown_connections 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0d3530 0x7f243c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243c0b6680 0x7f243c15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 --2- 172.21.15.106:0/79072458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f243c0b62a0 0x7f243c1f9f20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 >> 172.21.15.106:0/79072458 conn(0x7f243c075d10 msgr2=0x7f243c0c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 shutdown_connections 2024-04-26T21:46:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.239+0000 7f2441400640 1 -- 172.21.15.106:0/79072458 wait complete. 2024-04-26T21:46:04.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-26T21:46:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 msgr2=0x7f4a0406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 0x7f4a0406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c049a10 tx=0x7f4a10008040 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 msgr2=0x7f4a1c0ddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c3f0 tx=0x7f4a1800c8b0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 shutdown_connections 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a1c0e7a50 0x7f4a1c0dfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a1c0548a0 0x7f4a1c008dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4a0406c940 0x7f4a0406ee00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 --2- 172.21.15.106:0/2837828818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a1c03c360 0x7f4a1c0ddf20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.240+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 >> 172.21.15.106:0/2837828818 conn(0x7f4a1c056cd0 msgr2=0x7f4a1c03c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.241+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 shutdown_connections 2024-04-26T21:46:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:04.241+0000 7f4a2b000640 1 -- 172.21.15.106:0/2837828818 wait complete. 2024-04-26T21:46:04.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-26T21:46:04.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-26T21:46:04.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-26T21:46:04.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-26T21:46:04.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-04-26T21:46:05.229 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:46:05.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:46:05.242 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:46:05.246 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:46:05.251 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:46:05.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:46:05.264 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:46:05.266 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:46:05.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.514+0000 7f14fd400640 1 Processor -- start 2024-04-26T21:46:05.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.514+0000 7f14fd400640 1 -- start start 2024-04-26T21:46:05.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.516+0000 7f14fd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f80d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.517+0000 7f14fd400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f14f80d26a0 0x7f14f80d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.518+0000 7f14fd400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f14f80d3bd0 0x7f14f8163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.518+0000 7f14fd400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f14f80d4000 con 0x7f14f80d26a0 2024-04-26T21:46:05.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.518+0000 7f14fd400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f14f80cfd60 con 0x7f14f80d3bd0 2024-04-26T21:46:05.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.518+0000 7f14fd400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80d0040 con 0x7f14f815f450 2024-04-26T21:46:05.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.518+0000 7f14f6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f80d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.519+0000 7f14f6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f80d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39652/0 (socket says 172.21.15.106:39652) 2024-04-26T21:46:05.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.519+0000 7f14f6800640 1 -- 172.21.15.106:0/1698560400 learned_addr learned my addr 172.21.15.106:0/1698560400 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.519+0000 7f14f5e00640 1 --2- 172.21.15.106:0/1698560400 >> v2:172.21.15.106:3300/0 conn(0x7f14f80d26a0 0x7f14f80d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.519+0000 7f14f7200640 1 --2- 172.21.15.106:0/1698560400 >> v2:172.21.15.204:3300/0 conn(0x7f14f80d3bd0 0x7f14f8163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.521+0000 7f14f5e00640 1 -- 172.21.15.106:0/1698560400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 msgr2=0x7f14f80d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.521+0000 7f14f5e00640 1 --2- 172.21.15.106:0/1698560400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f80d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.521+0000 7f14f5e00640 1 -- 172.21.15.106:0/1698560400 >> v2:172.21.15.204:3300/0 conn(0x7f14f80d3bd0 msgr2=0x7f14f8163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.521+0000 7f14f5e00640 1 --2- 172.21.15.106:0/1698560400 >> v2:172.21.15.204:3300/0 conn(0x7f14f80d3bd0 0x7f14f8163000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.521+0000 7f14f5e00640 1 -- 172.21.15.106:0/1698560400 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f80d0320 con 0x7f14f80d26a0 2024-04-26T21:46:05.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.523+0000 7f14f5e00640 1 --2- 172.21.15.106:0/1698560400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d26a0 0x7f14f80d3690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14f80d3400 tx=0x7f14e802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d76a63adb5df695 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.523+0000 7f14f5400640 1 -- 172.21.15.106:0/1698560400 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8037070 con 0x7f14f80d26a0 2024-04-26T21:46:05.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.524+0000 7f14f5400640 1 -- 172.21.15.106:0/1698560400 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e8032440 con 0x7f14f80d26a0 2024-04-26T21:46:05.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.524+0000 7f14f5400640 1 -- 172.21.15.106:0/1698560400 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8032850 con 0x7f14f80d26a0 2024-04-26T21:46:05.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.525+0000 7f14fd400640 1 -- 172.21.15.106:0/1698560400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d26a0 msgr2=0x7f14f80d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.525+0000 7f14fd400640 1 --2- 172.21.15.106:0/1698560400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d26a0 0x7f14f80d3690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14f80d3400 tx=0x7f14e802a140 comp rx=0 tx=0).stop 2024-04-26T21:46:05.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 -- 172.21.15.106:0/1698560400 shutdown_connections 2024-04-26T21:46:05.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 --2- 172.21.15.106:0/1698560400 >> v2:172.21.15.204:3300/0 conn(0x7f14f80d3bd0 0x7f14f8163000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 --2- 172.21.15.106:0/1698560400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d26a0 0x7f14f80d3690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 --2- 172.21.15.106:0/1698560400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f80d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 -- 172.21.15.106:0/1698560400 >> 172.21.15.106:0/1698560400 conn(0x7f14f80d50b0 msgr2=0x7f14f80d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.526+0000 7f14fd400640 1 -- 172.21.15.106:0/1698560400 shutdown_connections 2024-04-26T21:46:05.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.527+0000 7f14fd400640 1 -- 172.21.15.106:0/1698560400 wait complete. 2024-04-26T21:46:05.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.530+0000 7f14fd400640 1 Processor -- start 2024-04-26T21:46:05.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.531+0000 7f14fd400640 1 -- start start 2024-04-26T21:46:05.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.533+0000 7f14fd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 0x7f14f814a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.533+0000 7f14fd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 0x7f14f80ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.533+0000 7f14fd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14f6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 0x7f14f814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14f6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 0x7f14f814a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37626/0 (socket says 172.21.15.106:37626) 2024-04-26T21:46:05.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14f6800640 1 -- 172.21.15.106:0/2095181010 learned_addr learned my addr 172.21.15.106:0/2095181010 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14f815f830 con 0x7f14f80d3bd0 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80d08e0 con 0x7f14f80d26a0 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80d5e00 con 0x7f14f815f450 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.534+0000 7f14f7200640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.535+0000 7f14f5e00640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 0x7f14f80ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.535+0000 7f14f7200640 1 -- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 msgr2=0x7f14f814a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.535+0000 7f14f7200640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 0x7f14f814a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.535+0000 7f14f7200640 1 -- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 msgr2=0x7f14f80ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.536+0000 7f14f7200640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 0x7f14f80ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.536+0000 7f14f7200640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f815a2e0 con 0x7f14f815f450 2024-04-26T21:46:05.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.536+0000 7f14f5e00640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 0x7f14f80ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:05.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.536+0000 7f14f7200640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f814ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14f000eae0 tx=0x7f14f000b0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.538+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0018020 con 0x7f14f815f450 2024-04-26T21:46:05.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.538+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f000bc10 con 0x7f14f815f450 2024-04-26T21:46:05.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.538+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0014400 con 0x7f14f815f450 2024-04-26T21:46:05.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.538+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f815a5c0 con 0x7f14f815f450 2024-04-26T21:46:05.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.538+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f80b9b20 con 0x7f14f815f450 2024-04-26T21:46:05.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.539+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f8069540 con 0x7f14f815f450 2024-04-26T21:46:05.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.543+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14f0005d10 con 0x7f14f815f450 2024-04-26T21:46:05.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.544+0000 7f14eea00640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 0x7f14d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.545+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14f008c840 con 0x7f14f815f450 2024-04-26T21:46:05.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.549+0000 7f14f6800640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 0x7f14d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.550+0000 7f14f6800640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 0x7f14d806ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14e40094a0 tx=0x7f14e4009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.554+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14f0022030 con 0x7f14f815f450 2024-04-26T21:46:05.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.557+0000 7f3fc8e00640 1 Processor -- start 2024-04-26T21:46:05.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.557+0000 7f3fc8e00640 1 -- start start 2024-04-26T21:46:05.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f3fc8e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3fc415f5f0 0x7f3fc40d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f3fc8e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 0x7f3fc40d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f2c17c00640 1 Processor -- start 2024-04-26T21:46:05.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f2c17c00640 1 -- start start 2024-04-26T21:46:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f3fc8e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc40d5e20 con 0x7f3fc40b3f80 2024-04-26T21:46:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f3fc8e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc40d5bd0 con 0x7f3fc40d2e90 2024-04-26T21:46:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.559+0000 7f3fc8e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc40d11e0 con 0x7f3fc415f5f0 2024-04-26T21:46:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.560+0000 7f3fc1400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 0x7f3fc40d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.560+0000 7f3fc1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3fc415f5f0 0x7f3fc40d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.560+0000 7f3fc1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3fc415f5f0 0x7f3fc40d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37638/0 (socket says 172.21.15.106:37638) 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.560+0000 7f3fc1e00640 1 -- 172.21.15.106:0/3594294856 learned_addr learned my addr 172.21.15.106:0/3594294856 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f3fc1e00640 1 -- 172.21.15.106:0/3594294856 >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 msgr2=0x7f3fc40d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f2c17c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c100b4100 0x7f2c100b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f2c17c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c100aeb60 con 0x7f2c100ae160 2024-04-26T21:46:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f2c17c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c10072440 con 0x7f2c100add80 2024-04-26T21:46:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f2c17c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2c10073050 con 0x7f2c100b4100 2024-04-26T21:46:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.561+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/3594294856 >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 0x7f3fc40d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.562+0000 7f3fc1e00640 1 -- 172.21.15.106:0/3594294856 >> v1:172.21.15.204:6789/0 conn(0x7f3fc40b3f80 legacy=0x7f3fc40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.562+0000 7f2c15200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2c100add80 0x7f2c10160130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:38384/0 (socket says 172.21.15.106:38384) 2024-04-26T21:46:05.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.562+0000 7f2c15200640 1 -- 172.21.15.106:0/1144254396 learned_addr learned my addr 172.21.15.106:0/1144254396 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:05.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.562+0000 7f3fc1e00640 1 -- 172.21.15.106:0/3594294856 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc40d14c0 con 0x7f3fc415f5f0 2024-04-26T21:46:05.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.562+0000 7f3fc1400640 1 --2- 172.21.15.106:0/3594294856 >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 0x7f3fc40d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.563+0000 7f2c15c00640 1 --2- 172.21.15.106:0/1144254396 >> v2:172.21.15.106:3300/0 conn(0x7f2c100b4100 0x7f2c100b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.563+0000 7f2c0f400640 1 -- 172.21.15.106:0/1144254396 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85878430 0 0) 0x7f2c10072440 con 0x7f2c100add80 2024-04-26T21:46:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.563+0000 7f2c0f400640 1 -- 172.21.15.106:0/1144254396 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf4003520 con 0x7f2c100add80 2024-04-26T21:46:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.564+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/3594294856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc415f5f0 0x7f3fc40d1930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fc408f900 tx=0x7f3fbc02a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecc42718d0e09cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.564+0000 7f2c15c00640 1 -- 172.21.15.106:0/1144254396 >> v1:172.21.15.204:6789/0 conn(0x7f2c100add80 legacy=0x7f2c10160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.564+0000 7f2c15c00640 1 -- 172.21.15.106:0/1144254396 >> v1:172.21.15.106:6789/0 conn(0x7f2c100ae160 legacy=0x7f2c100734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.564+0000 7f2c15c00640 1 -- 172.21.15.106:0/1144254396 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c10072720 con 0x7f2c100b4100 2024-04-26T21:46:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.564+0000 7f3fc0a00640 1 -- 172.21.15.106:0/3594294856 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc036070 con 0x7f3fc415f5f0 2024-04-26T21:46:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.565+0000 7f3fc0a00640 1 -- 172.21.15.106:0/3594294856 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fbc031600 con 0x7f3fc415f5f0 2024-04-26T21:46:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.565+0000 7f3fc0a00640 1 -- 172.21.15.106:0/3594294856 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0319d0 con 0x7f3fc415f5f0 2024-04-26T21:46:05.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.565+0000 7f2c15c00640 1 --2- 172.21.15.106:0/1144254396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100b4100 0x7f2c100b4550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097a0 tx=0x7f2c0002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64653714d0a4d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 -- 172.21.15.106:0/3594294856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc415f5f0 msgr2=0x7f3fc40d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/3594294856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc415f5f0 0x7f3fc40d1930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fc408f900 tx=0x7f3fbc02a640 comp rx=0 tx=0).stop 2024-04-26T21:46:05.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 -- 172.21.15.106:0/3594294856 shutdown_connections 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/3594294856 >> v2:172.21.15.106:3300/0 conn(0x7f3fc40d2e90 0x7f3fc40d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/3594294856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc415f5f0 0x7f3fc40d1930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 -- 172.21.15.106:0/3594294856 >> 172.21.15.106:0/3594294856 conn(0x7f3fc40d50b0 msgr2=0x7f3fc40d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f2c0f400640 1 -- 172.21.15.106:0/1144254396 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00037070 con 0x7f2c100b4100 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f2c0f400640 1 -- 172.21.15.106:0/1144254396 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c00032440 con 0x7f2c100b4100 2024-04-26T21:46:05.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f2c0f400640 1 -- 172.21.15.106:0/1144254396 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00037070 con 0x7f2c100b4100 2024-04-26T21:46:05.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 -- 172.21.15.106:0/3594294856 shutdown_connections 2024-04-26T21:46:05.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.566+0000 7f3fc8e00640 1 -- 172.21.15.106:0/3594294856 wait complete. 2024-04-26T21:46:05.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.568+0000 7f2c17c00640 1 -- 172.21.15.106:0/1144254396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100b4100 msgr2=0x7f2c100b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.568+0000 7f2c17c00640 1 --2- 172.21.15.106:0/1144254396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100b4100 0x7f2c100b4550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097a0 tx=0x7f2c0002a230 comp rx=0 tx=0).stop 2024-04-26T21:46:05.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.569+0000 7fd29e600640 1 Processor -- start 2024-04-26T21:46:05.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.570+0000 7f2c17c00640 1 -- 172.21.15.106:0/1144254396 shutdown_connections 2024-04-26T21:46:05.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.570+0000 7f2c17c00640 1 --2- 172.21.15.106:0/1144254396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100b4100 0x7f2c100b4550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.570+0000 7f2c17c00640 1 -- 172.21.15.106:0/1144254396 >> 172.21.15.106:0/1144254396 conn(0x7f2c100b3240 msgr2=0x7f2c100d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.569+0000 7fd29e600640 1 -- start start 2024-04-26T21:46:05.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.571+0000 7fd29e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd298072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.571+0000 7fd29e600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd29809eec0 con 0x7fd298070580 2024-04-26T21:46:05.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.570+0000 7f3fc8e00640 1 Processor -- start 2024-04-26T21:46:05.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.570+0000 7f3fc8e00640 1 -- start start 2024-04-26T21:46:05.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.571+0000 7fd29e600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2980730b0 con 0x7fd2980d1350 2024-04-26T21:46:05.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.571+0000 7fd29e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd298073950 con 0x7fd2980aee10 2024-04-26T21:46:05.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.571+0000 7f2c17c00640 1 -- 172.21.15.106:0/1144254396 shutdown_connections 2024-04-26T21:46:05.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7fd297400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd2980d1350 0x7fd2980ae700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:38388/0 (socket says 172.21.15.106:38388) 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 0x7f3fc40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 0x7f3fc4161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40b5dc0 con 0x7f3fc40b3f80 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40b5c40 con 0x7f3fc40d2e90 2024-04-26T21:46:05.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc40b5f40 con 0x7f3fc415f5f0 2024-04-26T21:46:05.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7fd297400640 1 -- 172.21.15.106:0/1311065829 learned_addr learned my addr 172.21.15.106:0/1311065829 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:05.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7fd296a00640 1 --2- 172.21.15.106:0/1311065829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd298072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f2c17c00640 1 -- 172.21.15.106:0/1144254396 wait complete. 2024-04-26T21:46:05.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.572+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37652/0 (socket says 172.21.15.106:37652) 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7f3fc1400640 1 -- 172.21.15.106:0/998611231 learned_addr learned my addr 172.21.15.106:0/998611231 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7f3fc2800640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 0x7f3fc4161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671631488 0 0) 0x7fd2980730b0 con 0x7fd2980d1350 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd27c003520 con 0x7fd2980d1350 2024-04-26T21:46:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 0x7f3fc40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296a00640 1 -- 172.21.15.106:0/1311065829 >> v1:172.21.15.204:6789/0 conn(0x7fd2980d1350 legacy=0x7fd2980ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296a00640 1 -- 172.21.15.106:0/1311065829 >> v1:172.21.15.106:6789/0 conn(0x7fd298070580 legacy=0x7fd298166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156299847 0 0) 0x7fd29809eec0 con 0x7fd298070580 2024-04-26T21:46:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.573+0000 7fd296a00640 1 -- 172.21.15.106:0/1311065829 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd298073390 con 0x7fd2980aee10 2024-04-26T21:46:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f3fc1400640 1 -- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 msgr2=0x7f3fc4161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f3fc1400640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 0x7f3fc4161830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f3fc1400640 1 -- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 msgr2=0x7f3fc40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f2c17c00640 1 Processor -- start 2024-04-26T21:46:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7feb4cc00640 1 Processor -- start 2024-04-26T21:46:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f3fc1400640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 0x7f3fc40cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.574+0000 7f3fc1400640 1 -- 172.21.15.106:0/998611231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc40cd100 con 0x7f3fc40d2e90 2024-04-26T21:46:05.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f3fc2800640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 0x7f3fc4161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:05.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7feb4cc00640 1 -- start start 2024-04-26T21:46:05.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 0x7f3fc40cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:05.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7fd296a00640 1 --2- 172.21.15.106:0/1311065829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd298072a20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd298073ad0 tx=0x7fd28802c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43010395e321d50a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f1e18e00640 1 Processor -- start 2024-04-26T21:46:05.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7f1e18e00640 1 -- start start 2024-04-26T21:46:05.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f3fc1400640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ebe0 tx=0x7f3fb400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f2c17c00640 1 -- start start 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100ae160 0x7f2c10162ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100b4100 0x7f2c10163400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c100ae610 con 0x7f2c100ae160 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10072ce0 con 0x7f2c100b4100 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c17c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c10159a70 con 0x7f2c100add80 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7f2c15200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2c15200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39700/0 (socket says 172.21.15.106:39700) 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2c15200640 1 -- 172.21.15.106:0/110493401 learned_addr learned my addr 172.21.15.106:0/110493401 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2c0fe00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100ae160 0x7f2c10162ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288039070 con 0x7fd2980aee10 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288034640 con 0x7fd2980aee10 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7fd296000640 1 -- 172.21.15.106:0/1311065829 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880349f0 con 0x7fd2980aee10 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7fd29e600640 1 -- 172.21.15.106:0/1311065829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 msgr2=0x7fd298072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7fd29e600640 1 --2- 172.21.15.106:0/1311065829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd298072a20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd298073ad0 tx=0x7fd28802c370 comp rx=0 tx=0).stop 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7fd29e600640 1 -- 172.21.15.106:0/1311065829 shutdown_connections 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7fd29e600640 1 --2- 172.21.15.106:0/1311065829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd298072a20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7fd29e600640 1 -- 172.21.15.106:0/1311065829 >> 172.21.15.106:0/1311065829 conn(0x7fd2980b3240 msgr2=0x7fd2980d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7fd29e600640 1 -- 172.21.15.106:0/1311065829 shutdown_connections 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7feb4cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 0x7feb4815f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7feb4cc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb480b5d30 con 0x7feb480d34a0 2024-04-26T21:46:05.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7feb4cc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb480b6010 con 0x7feb4815ef60 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7feb4cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb480b62f0 con 0x7feb480d5d90 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f2682000640 1 Processor -- start 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.575+0000 7f2682000640 1 -- start start 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f267c0c8330 0x7f267c0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 0x7f267c0b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0d2410 con 0x7f267c0c8330 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0d02b0 con 0x7f267c0d57f0 2024-04-26T21:46:05.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2682000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0d0590 con 0x7f267c0d1af0 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4017070 con 0x7f3fc40d2e90 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb400cc30 con 0x7f3fc40d2e90 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.576+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4013470 con 0x7f3fc40d2e90 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc40cd3e0 con 0x7f3fc40d2e90 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f1e18e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e1406ce40 0x7f1e1415c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f1e18e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1415c540 con 0x7f1e1406f310 2024-04-26T21:46:05.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f1e18e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e140723e0 con 0x7f1e14073860 2024-04-26T21:46:05.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f1e18e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1e140b4a10 con 0x7f1e1406ce40 2024-04-26T21:46:05.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7fd29e600640 1 -- 172.21.15.106:0/1311065829 wait complete. 2024-04-26T21:46:05.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc4160ce0 con 0x7f3fc40d2e90 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f267b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f267c0c8330 0x7f267c0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f267be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 0x7f267c0b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f267c0c8330 0x7f267c0d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52262/0 (socket says 172.21.15.106:52262) 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 -- 172.21.15.106:0/3506744885 learned_addr learned my addr 172.21.15.106:0/3506744885 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f267be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 0x7f267c0b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37670/0 (socket says 172.21.15.106:37670) 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7f2c15c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100b4100 0x7f2c10163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2c15200640 1 -- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100b4100 msgr2=0x7f2c10163400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2c15200640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100b4100 0x7f2c10163400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2c15200640 1 -- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100ae160 msgr2=0x7f2c10162ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2c15200640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100ae160 0x7f2c10162ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2c15200640 1 -- 172.21.15.106:0/110493401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c102019a0 con 0x7f2c100add80 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7feb46800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feb480d34a0 0x7feb480d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58958/0 (socket says 172.21.15.106:58958) 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7feb46800640 1 -- 172.21.15.106:0/2657791871 learned_addr learned my addr 172.21.15.106:0/2657791871 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.577+0000 7feb45400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 0x7feb4815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.578+0000 7feb45e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7feb4815ef60 0x7feb480b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:38398/0 (socket says 172.21.15.106:38398) 2024-04-26T21:46:05.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f1e13400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e1406ce40 0x7f1e1415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f1e13e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1e1406f310 0x7f1e1406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58968/0 (socket says 172.21.15.106:58968) 2024-04-26T21:46:05.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f1e13e00640 1 -- 172.21.15.106:0/1545708794 learned_addr learned my addr 172.21.15.106:0/1545708794 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:05.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 -- 172.21.15.106:0/3506744885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0d1af0 msgr2=0x7f267c0d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 --2- 172.21.15.106:0/3506744885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f2c15200640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c530 tx=0x7f2bfc00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc018070 con 0x7f2c100add80 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0075a0 con 0x7f2c100add80 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc004a00 con 0x7f2c100add80 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 -- 172.21.15.106:0/3506744885 >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 msgr2=0x7f267c0b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 --2- 172.21.15.106:0/3506744885 >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 0x7f267c0b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.579+0000 7f2673e00640 1 -- 172.21.15.106:0/3506744885 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0d0870 con 0x7f267c0c8330 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636924111 0 0) 0x7feb480b6010 con 0x7feb4815ef60 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb2c003520 con 0x7feb4815ef60 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2605561192 0 0) 0x7feb480b5d30 con 0x7feb480d34a0 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb480b6010 con 0x7feb480d34a0 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7fd80e400640 1 Processor -- start 2024-04-26T21:46:05.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc4069540 con 0x7f3fc40d2e90 2024-04-26T21:46:05.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7fd29e600640 1 Processor -- start 2024-04-26T21:46:05.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7fd80e400640 1 -- start start 2024-04-26T21:46:05.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7feb45400640 1 -- 172.21.15.106:0/2657791871 >> v1:172.21.15.204:6789/0 conn(0x7feb4815ef60 legacy=0x7feb480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.580+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c10201c80 con 0x7f2c100add80 2024-04-26T21:46:05.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c10202240 con 0x7f2c100add80 2024-04-26T21:46:05.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7fd29e600640 1 -- start start 2024-04-26T21:46:05.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7feb45400640 1 -- 172.21.15.106:0/2657791871 >> v1:172.21.15.106:6789/0 conn(0x7feb480d34a0 legacy=0x7feb480d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 630721704 0 0) 0x7feb480b6010 con 0x7feb480d34a0 2024-04-26T21:46:05.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.581+0000 7f2673e00640 1 --2- 172.21.15.106:0/3506744885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0c8330 0x7f267c0d1070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f267c168320 tx=0x7f266402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d90197c68db4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7feb45400640 1 -- 172.21.15.106:0/2657791871 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb480b65d0 con 0x7feb480d5d90 2024-04-26T21:46:05.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f1e12000640 1 -- 172.21.15.106:0/1545708794 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535778001 0 0) 0x7f1e04001110 con 0x7f1e1406f310 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f267aa00640 1 -- 172.21.15.106:0/3506744885 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664037070 con 0x7f267c0c8330 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f267aa00640 1 -- 172.21.15.106:0/3506744885 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2664032440 con 0x7f267c0c8330 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f267aa00640 1 -- 172.21.15.106:0/3506744885 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664032810 con 0x7f267c0c8330 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f1e13400640 1 -- 172.21.15.106:0/1545708794 >> v1:172.21.15.204:6789/0 conn(0x7f1e14073860 legacy=0x7f1e14072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f1e13400640 1 -- 172.21.15.106:0/1545708794 >> v1:172.21.15.106:6789/0 conn(0x7f1e1406f310 legacy=0x7f1e1406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.582+0000 7f1e13400640 1 -- 172.21.15.106:0/1545708794 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14073d70 con 0x7f1e1406ce40 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd80e400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd80815dc80 0x7fd808167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd80e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd808164240 0x7fd80806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd80e400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd808164740 con 0x7fd80806c770 2024-04-26T21:46:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd80e400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd8080b5ff0 con 0x7fd80815dc80 2024-04-26T21:46:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd80e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd80806d110 con 0x7fd808164240 2024-04-26T21:46:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd29e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 0x7fd298145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd29e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd29806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7f2682000640 1 -- 172.21.15.106:0/3506744885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0c8330 msgr2=0x7f267c0d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7f2682000640 1 --2- 172.21.15.106:0/3506744885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0c8330 0x7f267c0d1070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f267c168320 tx=0x7f266402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.583+0000 7fd29e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 0x7fd298156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd29e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd29815b160 con 0x7fd2980d1350 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd29e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd2980b4670 con 0x7fd298070580 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd29e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd29815b2e0 con 0x7fd2980aee10 2024-04-26T21:46:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2682000640 1 -- 172.21.15.106:0/3506744885 shutdown_connections 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2682000640 1 --2- 172.21.15.106:0/3506744885 >> v2:172.21.15.204:3300/0 conn(0x7f267c0d57f0 0x7f267c0b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2682000640 1 --2- 172.21.15.106:0/3506744885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0c8330 0x7f267c0d1070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2682000640 1 --2- 172.21.15.106:0/3506744885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2682000640 1 -- 172.21.15.106:0/3506744885 >> 172.21.15.106:0/3506744885 conn(0x7f267c09e3b0 msgr2=0x7f267c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd297400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 0x7fd298145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd807400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd80815dc80 0x7fd808167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd807400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd80815dc80 0x7fd808167c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37678/0 (socket says 172.21.15.106:37678) 2024-04-26T21:46:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd807400640 1 -- 172.21.15.106:0/2970428240 learned_addr learned my addr 172.21.15.106:0/2970428240 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd297400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 0x7fd298145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37682/0 (socket says 172.21.15.106:37682) 2024-04-26T21:46:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd297400640 1 -- 172.21.15.106:0/3879136090 learned_addr learned my addr 172.21.15.106:0/3879136090 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd297e00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 0x7fd298156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c10069540 con 0x7f2c100add80 2024-04-26T21:46:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7fd807e00640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd808164240 0x7fd80806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.584+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fb401a070 con 0x7f3fc40d2e90 2024-04-26T21:46:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7f2682000640 1 -- 172.21.15.106:0/3506744885 shutdown_connections 2024-04-26T21:46:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7f2682000640 1 -- 172.21.15.106:0/3506744885 wait complete. 2024-04-26T21:46:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd296a00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd29806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd807400640 1 -- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd808164240 msgr2=0x7fd80806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd807400640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd808164240 0x7fd80806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd807400640 1 -- 172.21.15.106:0/2970428240 >> v1:172.21.15.204:6789/0 conn(0x7fd80806c770 legacy=0x7fd808163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd807400640 1 -- 172.21.15.106:0/2970428240 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081671a0 con 0x7fd80815dc80 2024-04-26T21:46:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7feb45400640 1 --2- 172.21.15.106:0/2657791871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 0x7feb4815f340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb380097d0 tx=0x7feb3802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42522bd9386cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7f1e13400640 1 --2- 172.21.15.106:0/1545708794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e1415c000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e140725b0 tx=0x7f1e0c02c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eacef44c46d944f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd297e00640 1 -- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 msgr2=0x7fd29806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd297e00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd29806ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd297e00640 1 -- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 msgr2=0x7fd298145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd297e00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 0x7fd298145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd297e00640 1 -- 172.21.15.106:0/3879136090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29809bee0 con 0x7fd2980d1350 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.585+0000 7fd296a00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd29806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7f3fba000640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 0x7f3fa406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd297e00640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 0x7fd298156730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd290002230 tx=0x7fd29000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3fb408c1f0 con 0x7f3fc40d2e90 2024-04-26T21:46:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bfc004ba0 con 0x7f2c100add80 2024-04-26T21:46:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290017070 con 0x7fd2980d1350 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd290003da0 con 0x7fd2980d1350 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900052b0 con 0x7fd2980d1350 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29809c1c0 con 0x7fd2980d1350 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.586+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29809c780 con 0x7fd2980d1350 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.587+0000 7fd807400640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd80815dc80 0x7fd808167c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80098a0 tx=0x7fd7f802a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21177474915fec6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.587+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 0x7f3fa406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.587+0000 7f3fc1e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 0x7f3fa406ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc033000 tx=0x7f3fbc009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7f2682000640 1 Processor -- start 2024-04-26T21:46:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7f2c0cc00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29809cb20 con 0x7fd2980d1350 2024-04-26T21:46:05.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7f2682000640 1 -- start start 2024-04-26T21:46:05.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7f2c0fe00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.588+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc08ae00 con 0x7f2c100add80 2024-04-26T21:46:05.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2c0fe00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c10162960 tx=0x7f2c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd29001a070 con 0x7fd2980d1350 2024-04-26T21:46:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0c8330 0x7f267c1f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 0x7f267c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0cff40 con 0x7f267c0d57f0 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0b6960 con 0x7f267c0d1af0 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.589+0000 7f2682000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0d15b0 con 0x7f267c0c8330 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52278/0 (socket says 172.21.15.106:52278) 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 -- 172.21.15.106:0/2999975584 learned_addr learned my addr 172.21.15.106:0/2999975584 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267b400640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0c8330 0x7f267c1f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f2673e00640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 0x7f267c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 -- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0c8330 msgr2=0x7f267c1f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0c8330 0x7f267c1f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7fd28f400640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 0x7fd27806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd29008b140 con 0x7fd2980d1350 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 -- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 msgr2=0x7f267c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 0x7f267c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f267be00640 1 -- 172.21.15.106:0/2999975584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0b7ee0 con 0x7f267c0d57f0 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.590+0000 7f2673e00640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 0x7f267c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.591+0000 7fd297400640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 0x7fd27806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.591+0000 7f267be00640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2674007e60 tx=0x7f267400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.591+0000 7fd297400640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 0x7fd27806ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd284005880 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674014420 con 0x7f267c0d57f0 2024-04-26T21:46:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd290058f00 con 0x7fd2980d1350 2024-04-26T21:46:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674019070 con 0x7f267c0d57f0 2024-04-26T21:46:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c0b81c0 con 0x7f267c0d57f0 2024-04-26T21:46:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c06e640 con 0x7f267c0d57f0 2024-04-26T21:46:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.595+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674014820 con 0x7f267c0d57f0 2024-04-26T21:46:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.596+0000 7fd806000640 1 -- 172.21.15.106:0/2970428240 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8037070 con 0x7fd80815dc80 2024-04-26T21:46:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.596+0000 7f1e12000640 1 -- 172.21.15.106:0/1545708794 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c032770 con 0x7f1e1406ce40 2024-04-26T21:46:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.596+0000 7f1e12000640 1 -- 172.21.15.106:0/1545708794 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c038070 con 0x7f1e1406ce40 2024-04-26T21:46:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.596+0000 7f1e12000640 1 -- 172.21.15.106:0/1545708794 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c032b30 con 0x7f1e1406ce40 2024-04-26T21:46:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.597+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fb405a040 con 0x7f3fc40d2e90 2024-04-26T21:46:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.597+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2654005290 con 0x7f267c0d57f0 2024-04-26T21:46:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.597+0000 7fd806000640 1 -- 172.21.15.106:0/2970428240 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8032440 con 0x7fd80815dc80 2024-04-26T21:46:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.597+0000 7fd806000640 1 -- 172.21.15.106:0/2970428240 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8032810 con 0x7fd80815dc80 2024-04-26T21:46:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f1e18e00640 1 -- 172.21.15.106:0/1545708794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 msgr2=0x7f1e1415c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f1e18e00640 1 --2- 172.21.15.106:0/1545708794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e1415c000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e140725b0 tx=0x7f1e0c02c870 comp rx=0 tx=0).stop 2024-04-26T21:46:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f1e18e00640 1 -- 172.21.15.106:0/1545708794 shutdown_connections 2024-04-26T21:46:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f1e18e00640 1 --2- 172.21.15.106:0/1545708794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e1415c000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f1e18e00640 1 -- 172.21.15.106:0/1545708794 >> 172.21.15.106:0/1545708794 conn(0x7f1e1406c350 msgr2=0x7f1e1406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26740149c0 con 0x7f267c0d57f0 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 -- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd80815dc80 msgr2=0x7fd808167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd80815dc80 0x7fd808167c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80098a0 tx=0x7fd7f802a1f0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 -- 172.21.15.106:0/2970428240 shutdown_connections 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd808164240 0x7fd80806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 --2- 172.21.15.106:0/2970428240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd80815dc80 0x7fd808167c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.598+0000 7fd80e400640 1 -- 172.21.15.106:0/2970428240 >> 172.21.15.106:0/2970428240 conn(0x7fd8080b2d50 msgr2=0x7fd8080b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7fd80e400640 1 -- 172.21.15.106:0/2970428240 shutdown_connections 2024-04-26T21:46:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7f1e18e00640 1 -- 172.21.15.106:0/1545708794 shutdown_connections 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bfc058c50 con 0x7f2c100add80 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38037070 con 0x7feb480d5d90 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3802acc0 con 0x7feb480d5d90 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7feb44a00640 1 -- 172.21.15.106:0/2657791871 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340015f0 con 0x7feb480d5d90 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7f2673400640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 0x7f265806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f267408c220 con 0x7f267c0d57f0 2024-04-26T21:46:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7f1e18e00640 1 -- 172.21.15.106:0/1545708794 wait complete. 2024-04-26T21:46:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.600+0000 7f267b400640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 0x7f265806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.601+0000 7feb4cc00640 1 -- 172.21.15.106:0/2657791871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 msgr2=0x7feb4815f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.601+0000 7feb4cc00640 1 --2- 172.21.15.106:0/2657791871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 0x7feb4815f340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb380097d0 tx=0x7feb3802a230 comp rx=0 tx=0).stop 2024-04-26T21:46:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.599+0000 7fd80e400640 1 -- 172.21.15.106:0/2970428240 wait complete. 2024-04-26T21:46:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.602+0000 7f267b400640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 0x7f265806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f266c003610 tx=0x7f266c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.602+0000 7fd80e400640 1 Processor -- start 2024-04-26T21:46:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.602+0000 7fd80e400640 1 -- start start 2024-04-26T21:46:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7f1e18e00640 1 Processor -- start 2024-04-26T21:46:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7feb4cc00640 1 -- 172.21.15.106:0/2657791871 shutdown_connections 2024-04-26T21:46:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7feb4cc00640 1 --2- 172.21.15.106:0/2657791871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb480d5d90 0x7feb4815f340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7feb4cc00640 1 -- 172.21.15.106:0/2657791871 >> 172.21.15.106:0/2657791871 conn(0x7feb4809e3b0 msgr2=0x7feb480d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7feb4cc00640 1 -- 172.21.15.106:0/2657791871 shutdown_connections 2024-04-26T21:46:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7feb4cc00640 1 -- 172.21.15.106:0/2657791871 wait complete. 2024-04-26T21:46:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.603+0000 7f1e18e00640 1 -- start start 2024-04-26T21:46:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 0x7fd808201d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 0x7fd80809e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd808164a20 con 0x7fd80815dc80 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd808167760 con 0x7fd808164240 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.604+0000 7fd80e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd8080add80 con 0x7fd80806c770 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd807400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 0x7fd808201d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd807400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 0x7fd808201d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39744/0 (socket says 172.21.15.106:39744) 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14072960 con 0x7f1e14073860 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14072680 con 0x7f1e1406ce40 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7f1e18e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1406e3a0 con 0x7f1e1406f310 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd807e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd807e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37698/0 (socket says 172.21.15.106:37698) 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd807e00640 1 -- 172.21.15.106:0/2261704513 learned_addr learned my addr 172.21.15.106:0/2261704513 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.605+0000 7fd806a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 0x7fd80809e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e13e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e13e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52298/0 (socket says 172.21.15.106:52298) 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39752/0 (socket says 172.21.15.106:39752) 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:37702/0 (socket says 172.21.15.106:37702) 2024-04-26T21:46:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7f1e12a00640 1 -- 172.21.15.106:0/3718564942 learned_addr learned my addr 172.21.15.106:0/3718564942 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.606+0000 7feb4cc00640 1 Processor -- start 2024-04-26T21:46:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.607+0000 7feb4cc00640 1 -- start start 2024-04-26T21:46:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.607+0000 7fd807e00640 1 -- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 msgr2=0x7fd808201d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13400640 1 -- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 msgr2=0x7f1e140af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13400640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13400640 1 -- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 msgr2=0x7f1e1409daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13400640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13400640 1 -- 172.21.15.106:0/3718564942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1409e890 con 0x7f1e1406ce40 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7fd807e00640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 0x7fd808201d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7fd807e00640 1 -- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 msgr2=0x7fd80809e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7fd807e00640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 0x7fd80809e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7fd807e00640 1 -- 172.21.15.106:0/2261704513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80809e910 con 0x7fd808164240 2024-04-26T21:46:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e13e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f1e12a00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f267405a6c0 con 0x7f267c0d57f0 2024-04-26T21:46:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb480d34a0 0x7feb4815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7f1e13400640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c034000 tx=0x7f1e0c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4815ef60 0x7feb480b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.608+0000 7fd806a00640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 0x7fd80809e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:05.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7fd807e00640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd80000efc0 tx=0x7fd80000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feb480d3a80 con 0x7feb480d5d90 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb480d3900 con 0x7feb480d34a0 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb4cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb480d3c00 con 0x7feb4815ef60 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb45400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb45400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52314/0 (socket says 172.21.15.106:52314) 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.609+0000 7feb45400640 1 -- 172.21.15.106:0/935917791 learned_addr learned my addr 172.21.15.106:0/935917791 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:05.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c033950 con 0x7f1e1406ce40 2024-04-26T21:46:05.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb46800640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4815ef60 0x7feb480b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c033af0 con 0x7f1e1406ce40 2024-04-26T21:46:05.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0463f0 con 0x7f1e1406ce40 2024-04-26T21:46:05.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800018020 con 0x7fd808164240 2024-04-26T21:46:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd800014480 con 0x7fd808164240 2024-04-26T21:46:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800009e60 con 0x7fd808164240 2024-04-26T21:46:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e141f0f80 con 0x7f1e1406ce40 2024-04-26T21:46:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e141f1540 con 0x7f1e1406ce40 2024-04-26T21:46:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.611+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80809ba50 con 0x7fd808164240 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb45400640 1 -- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4815ef60 msgr2=0x7feb480b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb45400640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4815ef60 0x7feb480b9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb45400640 1 -- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb480d34a0 msgr2=0x7feb4815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb45400640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb480d34a0 0x7feb4815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.610+0000 7feb45400640 1 -- 172.21.15.106:0/935917791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb480cc760 con 0x7feb480d5d90 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.611+0000 7feb45400640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feb380098f0 tx=0x7feb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.611+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80809c010 con 0x7fd808164240 2024-04-26T21:46:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.612+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38037050 con 0x7feb480d5d90 2024-04-26T21:46:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.612+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38043070 con 0x7feb480d5d90 2024-04-26T21:46:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.612+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3803ca50 con 0x7feb480d5d90 2024-04-26T21:46:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.613+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd808069540 con 0x7fd808164240 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.613+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb480cca40 con 0x7feb480d5d90 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.613+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb480cd000 con 0x7feb480d5d90 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.613+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e140693c0 con 0x7f1e1406ce40 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.615+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e0c032070 con 0x7f1e1406ce40 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.617+0000 7f1e0b400640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.617+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e0c078cd0 con 0x7f1e1406ce40 2024-04-26T21:46:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.614+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb48069540 con 0x7feb480d5d90 2024-04-26T21:46:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.616+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb3803cbf0 con 0x7feb480d5d90 2024-04-26T21:46:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.618+0000 7f1e12a00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.619+0000 7f1e12a00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 0x7f1df406ee00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e1415b040 tx=0x7f1e00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.617+0000 7feb3e000640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 0x7feb2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.617+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb380ac5b0 con 0x7feb480d5d90 2024-04-26T21:46:05.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.619+0000 7feb45e00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 0x7feb2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.620+0000 7feb45e00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 0x7feb2806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb480c8160 tx=0x7feb40007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.619+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd800005d10 con 0x7fd808164240 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.621+0000 7fd7ff400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 0x7fd7e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.621+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd80008cdc0 con 0x7fd808164240 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.622+0000 7fd807400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 0x7fd7e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.623+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb3807aaf0 con 0x7feb480d5d90 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.623+0000 7fd807400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 0x7fd7e806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8034000 tx=0x7fd7f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.628+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e0c07ac30 con 0x7f1e1406ce40 2024-04-26T21:46:05.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.641+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd800022030 con 0x7fd808164240 2024-04-26T21:46:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.987+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f14f80d3470 con 0x7f14f815f450 2024-04-26T21:46:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.988+0000 7f14eea00640 1 -- 172.21.15.106:0/2095181010 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14f005ace0 con 0x7f14f815f450 2024-04-26T21:46:05.989 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:05.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.992+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 msgr2=0x7f14d806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.992+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 0x7f14d806ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14e40094a0 tx=0x7f14e4009240 comp rx=0 tx=0).stop 2024-04-26T21:46:05.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.992+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 msgr2=0x7f14f814ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.993+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f814ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14f000eae0 tx=0x7f14f000b0b0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.993+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 shutdown_connections 2024-04-26T21:46:05.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.993+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d806c940 0x7f14d806ee00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.993+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f815f450 0x7f14f814ca30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.993+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d3bd0 0x7f14f80ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.994+0000 7f14fd400640 1 --2- 172.21.15.106:0/2095181010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f80d26a0 0x7f14f814a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.994+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 >> 172.21.15.106:0/2095181010 conn(0x7f14f80d50b0 msgr2=0x7f14f80c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.994+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 shutdown_connections 2024-04-26T21:46:05.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:05.994+0000 7f14fd400640 1 -- 172.21.15.106:0/2095181010 wait complete. 2024-04-26T21:46:06.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-26T21:46:06.021 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.025+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2c100aef70 con 0x7f2c100add80 2024-04-26T21:46:06.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.026+0000 7f2c0cc00640 1 -- 172.21.15.106:0/110493401 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2bfc05cf50 con 0x7f2c100add80 2024-04-26T21:46:06.026 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:06.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.029+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 msgr2=0x7f2bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.029+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c10162960 tx=0x7f2c08009040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.029+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 msgr2=0x7f2c10156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.029+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c530 tx=0x7f2bfc00c9f0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 shutdown_connections 2024-04-26T21:46:06.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100b4100 0x7f2c10163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c100ae160 0x7f2c10162ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 --2- 172.21.15.106:0/110493401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c100add80 0x7f2c10156f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.030+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 >> 172.21.15.106:0/110493401 conn(0x7f2c100b3240 msgr2=0x7f2c1015ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.031+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 shutdown_connections 2024-04-26T21:46:06.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.031+0000 7f2c17c00640 1 -- 172.21.15.106:0/110493401 wait complete. 2024-04-26T21:46:06.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.033+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2654005580 con 0x7f267c0d57f0 2024-04-26T21:46:06.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.034+0000 7f2673400640 1 -- 172.21.15.106:0/2999975584 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2674059fc0 con 0x7f267c0d57f0 2024-04-26T21:46:06.034 INFO:teuthology.orchestra.run.smithi106.stdout:55834574853 2024-04-26T21:46:06.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.035+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd298069540 con 0x7fd2980d1350 2024-04-26T21:46:06.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.036+0000 7fd28f400640 1 -- 172.21.15.106:0/3879136090 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd29005d200 con 0x7fd2980d1350 2024-04-26T21:46:06.036 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 msgr2=0x7f265806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 0x7f265806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f266c003610 tx=0x7f266c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 msgr2=0x7f267c154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2674007e60 tx=0x7f267400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 shutdown_connections 2024-04-26T21:46:06.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f267c0d57f0 0x7f267c154140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267c0d1af0 0x7f267c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f265806c940 0x7f265806ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 --2- 172.21.15.106:0/2999975584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f267c0c8330 0x7f267c1f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.037+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 >> 172.21.15.106:0/2999975584 conn(0x7f267c09e3b0 msgr2=0x7f267c0d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.038+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 shutdown_connections 2024-04-26T21:46:06.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.038+0000 7f2682000640 1 -- 172.21.15.106:0/2999975584 wait complete. 2024-04-26T21:46:06.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 msgr2=0x7fd27806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 0x7fd27806ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd284005880 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 msgr2=0x7fd298156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 0x7fd298156730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd290002230 tx=0x7fd29000cbe0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 shutdown_connections 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2980d1350 0x7fd298156730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd27806c940 0x7fd27806ee00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2980aee10 0x7fd29806ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 --2- 172.21.15.106:0/3879136090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd298070580 0x7fd298145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 >> 172.21.15.106:0/3879136090 conn(0x7fd2980b3240 msgr2=0x7fd2980b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 shutdown_connections 2024-04-26T21:46:06.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.039+0000 7fd29e600640 1 -- 172.21.15.106:0/3879136090 wait complete. 2024-04-26T21:46:06.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.043+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3fc40baf30 con 0x7f3fc40d2e90 2024-04-26T21:46:06.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.044+0000 7f3fba000640 1 -- 172.21.15.106:0/998611231 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3fb405e340 con 0x7f3fc40d2e90 2024-04-26T21:46:06.045 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-26T21:46:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.048+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 msgr2=0x7f3fa406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.048+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 0x7f3fa406ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc033000 tx=0x7f3fbc009000 comp rx=0 tx=0).stop 2024-04-26T21:46:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.048+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 msgr2=0x7f3fc415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.048+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ebe0 tx=0x7f3fb400c520 comp rx=0 tx=0).stop 2024-04-26T21:46:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 shutdown_connections 2024-04-26T21:46:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3fa406c9c0 0x7f3fa406ee80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fc415f5f0 0x7f3fc4161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fc40d2e90 0x7f3fc415df60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 --2- 172.21.15.106:0/998611231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc40b3f80 0x7f3fc40cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 >> 172.21.15.106:0/998611231 conn(0x7f3fc40d50b0 msgr2=0x7f3fc40b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 shutdown_connections 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.049+0000 7f3fc8e00640 1 -- 172.21.15.106:0/998611231 wait complete. 2024-04-26T21:46:06.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.054+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feb4808d090 con 0x7feb480d5d90 2024-04-26T21:46:06.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.055+0000 7feb3e000640 1 -- 172.21.15.106:0/935917791 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb3807a3f0 con 0x7feb480d5d90 2024-04-26T21:46:06.055 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:06.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.058+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 msgr2=0x7feb2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.058+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 0x7feb2806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb480c8160 tx=0x7feb40007ca0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.058+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 msgr2=0x7feb480b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.058+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feb380098f0 tx=0x7feb380046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 shutdown_connections 2024-04-26T21:46:06.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb2806c8f0 0x7feb2806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb4815ef60 0x7feb480b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb480d5d90 0x7feb480b9a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 --2- 172.21.15.106:0/935917791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb480d34a0 0x7feb4815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 >> 172.21.15.106:0/935917791 conn(0x7feb4809e3b0 msgr2=0x7feb480b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 shutdown_connections 2024-04-26T21:46:06.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.059+0000 7feb4cc00640 1 -- 172.21.15.106:0/935917791 wait complete. 2024-04-26T21:46:06.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-26T21:46:06.061 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.061+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1e140b0610 con 0x7f1e1406ce40 2024-04-26T21:46:06.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-26T21:46:06.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.062+0000 7f1e0b400640 1 -- 172.21.15.106:0/3718564942 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e0c07ef30 con 0x7f1e1406ce40 2024-04-26T21:46:06.063 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-26T21:46:06.063 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:06.063 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.064+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 msgr2=0x7f1df406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.064+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 0x7f1df406ee00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e1415b040 tx=0x7f1e00009040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.064+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 msgr2=0x7f1e140aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.064+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c034000 tx=0x7f1e0c004ab0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 shutdown_connections 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e14073860 0x7f1e1409daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e1406f310 0x7f1e140af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 --2- 172.21.15.106:0/3718564942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1406ce40 0x7f1e140aedd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 >> 172.21.15.106:0/3718564942 conn(0x7f1e1406c350 msgr2=0x7f1e140b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 shutdown_connections 2024-04-26T21:46:06.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.065+0000 7f1e18e00640 1 -- 172.21.15.106:0/3718564942 wait complete. 2024-04-26T21:46:06.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.066+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd808073fa0 con 0x7fd808164240 2024-04-26T21:46:06.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.067+0000 7fd7ff400640 1 -- 172.21.15.106:0/2261704513 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd800010040 con 0x7fd808164240 2024-04-26T21:46:06.067 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 msgr2=0x7fd7e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 0x7fd7e806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8034000 tx=0x7fd7f802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 msgr2=0x7fd808072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd80000efc0 tx=0x7fd80000c490 comp rx=0 tx=0).stop 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 shutdown_connections 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd808164240 0x7fd808072280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd80815dc80 0x7fd80809e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd7e806c9c0 0x7fd7e806ee80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 --2- 172.21.15.106:0/2261704513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80806c770 0x7fd808201d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 >> 172.21.15.106:0/2261704513 conn(0x7fd8080b2d50 msgr2=0x7fd8080b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 shutdown_connections 2024-04-26T21:46:06.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.071+0000 7fd80e400640 1 -- 172.21.15.106:0/2261704513 wait complete. 2024-04-26T21:46:06.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-26T21:46:06.074 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-26T21:46:06.083 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-26T21:46:06.083 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-26T21:46:06.088 DEBUG:teuthology.parallel:result is None 2024-04-26T21:46:06.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:46:06.088 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:46:06.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.319+0000 7fee4b400640 1 Processor -- start 2024-04-26T21:46:06.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.319+0000 7fee4b400640 1 -- start start 2024-04-26T21:46:06.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee4b400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fee440d5d90 0x7fee4415f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee4b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d34a0 0x7fee440d3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee4b400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee440b5d30 con 0x7fee4415ef60 2024-04-26T21:46:06.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee4b400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fee440b6010 con 0x7fee440d5d90 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee4b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fee440b62f0 con 0x7fee440d34a0 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fee440d5d90 0x7fee4415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fee440d5d90 0x7fee4415f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52316/0 (socket says 172.21.15.106:52316) 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 -- 172.21.15.106:0/1989202890 learned_addr learned my addr 172.21.15.106:0/1989202890 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee49400640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d34a0 0x7fee440d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 -- 172.21.15.106:0/1989202890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d34a0 msgr2=0x7fee440d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d34a0 0x7fee440d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 -- 172.21.15.106:0/1989202890 >> v1:172.21.15.204:6789/0 conn(0x7fee4415ef60 legacy=0x7fee440b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.321+0000 7fee43e00640 1 -- 172.21.15.106:0/1989202890 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee440b65d0 con 0x7fee440d5d90 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.322+0000 7fee43e00640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee440d5d90 0x7fee4415f340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009e50 tx=0x7fee2c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bb476565ad488a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee43400640 1 -- 172.21.15.106:0/1989202890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c036070 con 0x7fee440d5d90 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee43400640 1 -- 172.21.15.106:0/1989202890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c031600 con 0x7fee440d5d90 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee43400640 1 -- 172.21.15.106:0/1989202890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0319d0 con 0x7fee440d5d90 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 -- 172.21.15.106:0/1989202890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee440d5d90 msgr2=0x7fee4415f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee440d5d90 0x7fee4415f340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009e50 tx=0x7fee2c02a750 comp rx=0 tx=0).stop 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 -- 172.21.15.106:0/1989202890 shutdown_connections 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d34a0 0x7fee440d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 --2- 172.21.15.106:0/1989202890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee440d5d90 0x7fee4415f340 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 -- 172.21.15.106:0/1989202890 >> 172.21.15.106:0/1989202890 conn(0x7fee4409e3b0 msgr2=0x7fee440d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.323+0000 7fee4b400640 1 -- 172.21.15.106:0/1989202890 shutdown_connections 2024-04-26T21:46:06.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.324+0000 7fee4b400640 1 -- 172.21.15.106:0/1989202890 wait complete. 2024-04-26T21:46:06.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.325+0000 7fee4b400640 1 Processor -- start 2024-04-26T21:46:06.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.325+0000 7fee4b400640 1 -- start start 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee440d34a0 0x7fee4415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d5d90 0x7fee441f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fee440d0350 con 0x7fee4415ef60 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee440d01d0 con 0x7fee440d34a0 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee4b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fee440d04d0 con 0x7fee440d5d90 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee49400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee49400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52318/0 (socket says 172.21.15.106:52318) 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee49400640 1 -- 172.21.15.106:0/2310927140 learned_addr learned my addr 172.21.15.106:0/2310927140 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.326+0000 7fee48a00640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee440d34a0 0x7fee4415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee43e00640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d5d90 0x7fee441f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 -- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d5d90 msgr2=0x7fee441f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d5d90 0x7fee441f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 -- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee440d34a0 msgr2=0x7fee4415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee440d34a0 0x7fee4415fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 -- 172.21.15.106:0/2310927140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee441602c0 con 0x7fee4415ef60 2024-04-26T21:46:06.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee49400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d600 tx=0x7fee3c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0125f0 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3c019070 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee441605a0 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0129f0 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.327+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee441f5b60 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.328+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee441f5d60 con 0x7fee4415ef60 2024-04-26T21:46:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.329+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee3c012b90 con 0x7fee4415ef60 2024-04-26T21:46:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.330+0000 7fee40c00640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 0x7fee2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.330+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee3c08bf40 con 0x7fee4415ef60 2024-04-26T21:46:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.330+0000 7fee48a00640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 0x7fee2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.331+0000 7fee48a00640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 0x7fee2006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee4406d720 tx=0x7fee34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.334+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee441f5d60 con 0x7fee4415ef60 2024-04-26T21:46:06.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.521+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee44069540 con 0x7fee2006c8f0 2024-04-26T21:46:06.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.539+0000 7fee40c00640 1 -- 172.21.15.106:0/2310927140 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fee44069540 con 0x7fee2006c8f0 2024-04-26T21:46:06.543 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:06.544 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 msgr2=0x7fee2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 0x7fee2006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee4406d720 tx=0x7fee34009040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 msgr2=0x7fee4415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d600 tx=0x7fee3c00dac0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 shutdown_connections 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee4415ef60 0x7fee4415f3b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fee2006c8f0 0x7fee2006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee440d5d90 0x7fee441f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 --2- 172.21.15.106:0/2310927140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee440d34a0 0x7fee4415fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 >> 172.21.15.106:0/2310927140 conn(0x7fee4409e3b0 msgr2=0x7fee440d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 shutdown_connections 2024-04-26T21:46:06.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.546+0000 7fee4b400640 1 -- 172.21.15.106:0/2310927140 wait complete. 2024-04-26T21:46:06.564 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T21:46:05.090837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.430104"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698213+0000","last_change":"2024-04-26T21:45:56.703198+0000","last_active":"2024-04-26T21:45:57.698213+0000","last_peered":"2024-04-26T21:45:57.698213+0000","last_clean":"2024-04-26T21:45:57.698213+0000","last_became_active":"2024-04-26T21:45:56.700438+0000","last_became_peered":"2024-04-26T21:45:56.700438+0000","last_unstale":"2024-04-26T21:45:57.698213+0000","last_undegraded":"2024-04-26T21:45:57.698213+0000","last_fullsized":"2024-04-26T21:45:57.698213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:44.744042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688536+0000","last_change":"2024-04-26T21:45:56.699815+0000","last_active":"2024-04-26T21:45:57.688536+0000","last_peered":"2024-04-26T21:45:57.688536+0000","last_clean":"2024-04-26T21:45:57.688536+0000","last_became_active":"2024-04-26T21:45:56.699515+0000","last_became_peered":"2024-04-26T21:45:56.699515+0000","last_unstale":"2024-04-26T21:45:57.688536+0000","last_undegraded":"2024-04-26T21:45:57.688536+0000","last_fullsized":"2024-04-26T21:45:57.688536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784161+0000","last_change":"2024-04-26T21:45:56.696080+0000","last_active":"2024-04-26T21:45:57.784161+0000","last_peered":"2024-04-26T21:45:57.784161+0000","last_clean":"2024-04-26T21:45:57.784161+0000","last_became_active":"2024-04-26T21:45:56.695767+0000","last_became_peered":"2024-04-26T21:45:56.695767+0000","last_unstale":"2024-04-26T21:45:57.784161+0000","last_undegraded":"2024-04-26T21:45:57.784161+0000","last_fullsized":"2024-04-26T21:45:57.784161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786723+0000","last_change":"2024-04-26T21:45:56.724164+0000","last_active":"2024-04-26T21:45:57.786723+0000","last_peered":"2024-04-26T21:45:57.786723+0000","last_clean":"2024-04-26T21:45:57.786723+0000","last_became_active":"2024-04-26T21:45:56.719445+0000","last_became_peered":"2024-04-26T21:45:56.719445+0000","last_unstale":"2024-04-26T21:45:57.786723+0000","last_undegraded":"2024-04-26T21:45:57.786723+0000","last_fullsized":"2024-04-26T21:45:57.786723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697154+0000","last_change":"2024-04-26T21:45:56.692330+0000","last_active":"2024-04-26T21:45:57.697154+0000","last_peered":"2024-04-26T21:45:57.697154+0000","last_clean":"2024-04-26T21:45:57.697154+0000","last_became_active":"2024-04-26T21:45:56.692002+0000","last_became_peered":"2024-04-26T21:45:56.692002+0000","last_unstale":"2024-04-26T21:45:57.697154+0000","last_undegraded":"2024-04-26T21:45:57.697154+0000","last_fullsized":"2024-04-26T21:45:57.697154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786129+0000","last_change":"2024-04-26T21:45:56.719030+0000","last_active":"2024-04-26T21:45:57.786129+0000","last_peered":"2024-04-26T21:45:57.786129+0000","last_clean":"2024-04-26T21:45:57.786129+0000","last_became_active":"2024-04-26T21:45:56.718334+0000","last_became_peered":"2024-04-26T21:45:56.718334+0000","last_unstale":"2024-04-26T21:45:57.786129+0000","last_undegraded":"2024-04-26T21:45:57.786129+0000","last_fullsized":"2024-04-26T21:45:57.786129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696797+0000","last_change":"2024-04-26T21:45:56.692710+0000","last_active":"2024-04-26T21:45:57.696797+0000","last_peered":"2024-04-26T21:45:57.696797+0000","last_clean":"2024-04-26T21:45:57.696797+0000","last_became_active":"2024-04-26T21:45:56.692374+0000","last_became_peered":"2024-04-26T21:45:56.692374+0000","last_unstale":"2024-04-26T21:45:57.696797+0000","last_undegraded":"2024-04-26T21:45:57.696797+0000","last_fullsized":"2024-04-26T21:45:57.696797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783131+0000","last_change":"2024-04-26T21:45:56.718800+0000","last_active":"2024-04-26T21:45:57.783131+0000","last_peered":"2024-04-26T21:45:57.783131+0000","last_clean":"2024-04-26T21:45:57.783131+0000","last_became_active":"2024-04-26T21:45:56.717337+0000","last_became_peered":"2024-04-26T21:45:56.717337+0000","last_unstale":"2024-04-26T21:45:57.783131+0000","last_undegraded":"2024-04-26T21:45:57.783131+0000","last_fullsized":"2024-04-26T21:45:57.783131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698512+0000","last_change":"2024-04-26T21:45:56.722478+0000","last_active":"2024-04-26T21:45:57.698512+0000","last_peered":"2024-04-26T21:45:57.698512+0000","last_clean":"2024-04-26T21:45:57.698512+0000","last_became_active":"2024-04-26T21:45:56.722182+0000","last_became_peered":"2024-04-26T21:45:56.722182+0000","last_unstale":"2024-04-26T21:45:57.698512+0000","last_undegraded":"2024-04-26T21:45:57.698512+0000","last_fullsized":"2024-04-26T21:45:57.698512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:22.652553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706561+0000","last_change":"2024-04-26T21:45:56.716505+0000","last_active":"2024-04-26T21:45:57.706561+0000","last_peered":"2024-04-26T21:45:57.706561+0000","last_clean":"2024-04-26T21:45:57.706561+0000","last_became_active":"2024-04-26T21:45:56.716200+0000","last_became_peered":"2024-04-26T21:45:56.716200+0000","last_unstale":"2024-04-26T21:45:57.706561+0000","last_undegraded":"2024-04-26T21:45:57.706561+0000","last_fullsized":"2024-04-26T21:45:57.706561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706119+0000","last_change":"2024-04-26T21:45:56.719125+0000","last_active":"2024-04-26T21:45:57.706119+0000","last_peered":"2024-04-26T21:45:57.706119+0000","last_clean":"2024-04-26T21:45:57.706119+0000","last_became_active":"2024-04-26T21:45:56.718753+0000","last_became_peered":"2024-04-26T21:45:56.718753+0000","last_unstale":"2024-04-26T21:45:57.706119+0000","last_undegraded":"2024-04-26T21:45:57.706119+0000","last_fullsized":"2024-04-26T21:45:57.706119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694414+0000","last_change":"2024-04-26T21:45:56.703498+0000","last_active":"2024-04-26T21:45:57.694414+0000","last_peered":"2024-04-26T21:45:57.694414+0000","last_clean":"2024-04-26T21:45:57.694414+0000","last_became_active":"2024-04-26T21:45:56.702682+0000","last_became_peered":"2024-04-26T21:45:56.702682+0000","last_unstale":"2024-04-26T21:45:57.694414+0000","last_undegraded":"2024-04-26T21:45:57.694414+0000","last_fullsized":"2024-04-26T21:45:57.694414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783918+0000","last_change":"2024-04-26T21:45:56.690293+0000","last_active":"2024-04-26T21:45:57.783918+0000","last_peered":"2024-04-26T21:45:57.783918+0000","last_clean":"2024-04-26T21:45:57.783918+0000","last_became_active":"2024-04-26T21:45:56.689962+0000","last_became_peered":"2024-04-26T21:45:56.689962+0000","last_unstale":"2024-04-26T21:45:57.783918+0000","last_undegraded":"2024-04-26T21:45:57.783918+0000","last_fullsized":"2024-04-26T21:45:57.783918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783643+0000","last_change":"2024-04-26T21:45:56.701473+0000","last_active":"2024-04-26T21:45:57.783643+0000","last_peered":"2024-04-26T21:45:57.783643+0000","last_clean":"2024-04-26T21:45:57.783643+0000","last_became_active":"2024-04-26T21:45:56.697858+0000","last_became_peered":"2024-04-26T21:45:56.697858+0000","last_unstale":"2024-04-26T21:45:57.783643+0000","last_undegraded":"2024-04-26T21:45:57.783643+0000","last_fullsized":"2024-04-26T21:45:57.783643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:13.204924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687846+0000","last_change":"2024-04-26T21:45:56.684598+0000","last_active":"2024-04-26T21:45:57.687846+0000","last_peered":"2024-04-26T21:45:57.687846+0000","last_clean":"2024-04-26T21:45:57.687846+0000","last_became_active":"2024-04-26T21:45:56.684065+0000","last_became_peered":"2024-04-26T21:45:56.684065+0000","last_unstale":"2024-04-26T21:45:57.687846+0000","last_undegraded":"2024-04-26T21:45:57.687846+0000","last_fullsized":"2024-04-26T21:45:57.687846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785651+0000","last_change":"2024-04-26T21:45:56.713842+0000","last_active":"2024-04-26T21:45:57.785651+0000","last_peered":"2024-04-26T21:45:57.785651+0000","last_clean":"2024-04-26T21:45:57.785651+0000","last_became_active":"2024-04-26T21:45:56.713554+0000","last_became_peered":"2024-04-26T21:45:56.713554+0000","last_unstale":"2024-04-26T21:45:57.785651+0000","last_undegraded":"2024-04-26T21:45:57.785651+0000","last_fullsized":"2024-04-26T21:45:57.785651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.787120+0000","last_change":"2024-04-26T21:45:56.717288+0000","last_active":"2024-04-26T21:45:57.787120+0000","last_peered":"2024-04-26T21:45:57.787120+0000","last_clean":"2024-04-26T21:45:57.787120+0000","last_became_active":"2024-04-26T21:45:56.716846+0000","last_became_peered":"2024-04-26T21:45:56.716846+0000","last_unstale":"2024-04-26T21:45:57.787120+0000","last_undegraded":"2024-04-26T21:45:57.787120+0000","last_fullsized":"2024-04-26T21:45:57.787120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782583+0000","last_change":"2024-04-26T21:45:56.709803+0000","last_active":"2024-04-26T21:45:57.782583+0000","last_peered":"2024-04-26T21:45:57.782583+0000","last_clean":"2024-04-26T21:45:57.782583+0000","last_became_active":"2024-04-26T21:45:56.709518+0000","last_became_peered":"2024-04-26T21:45:56.709518+0000","last_unstale":"2024-04-26T21:45:57.782583+0000","last_undegraded":"2024-04-26T21:45:57.782583+0000","last_fullsized":"2024-04-26T21:45:57.782583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697786+0000","last_change":"2024-04-26T21:45:56.691493+0000","last_active":"2024-04-26T21:45:57.697786+0000","last_peered":"2024-04-26T21:45:57.697786+0000","last_clean":"2024-04-26T21:45:57.697786+0000","last_became_active":"2024-04-26T21:45:56.691171+0000","last_became_peered":"2024-04-26T21:45:56.691171+0000","last_unstale":"2024-04-26T21:45:57.697786+0000","last_undegraded":"2024-04-26T21:45:57.697786+0000","last_fullsized":"2024-04-26T21:45:57.697786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706090+0000","last_change":"2024-04-26T21:45:56.696933+0000","last_active":"2024-04-26T21:45:57.706090+0000","last_peered":"2024-04-26T21:45:57.706090+0000","last_clean":"2024-04-26T21:45:57.706090+0000","last_became_active":"2024-04-26T21:45:56.696509+0000","last_became_peered":"2024-04-26T21:45:56.696509+0000","last_unstale":"2024-04-26T21:45:57.706090+0000","last_undegraded":"2024-04-26T21:45:57.706090+0000","last_fullsized":"2024-04-26T21:45:57.706090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784228+0000","last_change":"2024-04-26T21:45:56.696412+0000","last_active":"2024-04-26T21:45:57.784228+0000","last_peered":"2024-04-26T21:45:57.784228+0000","last_clean":"2024-04-26T21:45:57.784228+0000","last_became_active":"2024-04-26T21:45:56.695758+0000","last_became_peered":"2024-04-26T21:45:56.695758+0000","last_unstale":"2024-04-26T21:45:57.784228+0000","last_undegraded":"2024-04-26T21:45:57.784228+0000","last_fullsized":"2024-04-26T21:45:57.784228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786146+0000","last_change":"2024-04-26T21:45:56.717913+0000","last_active":"2024-04-26T21:45:57.786146+0000","last_peered":"2024-04-26T21:45:57.786146+0000","last_clean":"2024-04-26T21:45:57.786146+0000","last_became_active":"2024-04-26T21:45:56.717579+0000","last_became_peered":"2024-04-26T21:45:56.717579+0000","last_unstale":"2024-04-26T21:45:57.786146+0000","last_undegraded":"2024-04-26T21:45:57.786146+0000","last_fullsized":"2024-04-26T21:45:57.786146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783112+0000","last_change":"2024-04-26T21:45:56.694204+0000","last_active":"2024-04-26T21:45:57.783112+0000","last_peered":"2024-04-26T21:45:57.783112+0000","last_clean":"2024-04-26T21:45:57.783112+0000","last_became_active":"2024-04-26T21:45:56.693845+0000","last_became_peered":"2024-04-26T21:45:56.693845+0000","last_unstale":"2024-04-26T21:45:57.783112+0000","last_undegraded":"2024-04-26T21:45:57.783112+0000","last_fullsized":"2024-04-26T21:45:57.783112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785959+0000","last_change":"2024-04-26T21:45:56.710113+0000","last_active":"2024-04-26T21:45:57.785959+0000","last_peered":"2024-04-26T21:45:57.785959+0000","last_clean":"2024-04-26T21:45:57.785959+0000","last_became_active":"2024-04-26T21:45:56.709659+0000","last_became_peered":"2024-04-26T21:45:56.709659+0000","last_unstale":"2024-04-26T21:45:57.785959+0000","last_undegraded":"2024-04-26T21:45:57.785959+0000","last_fullsized":"2024-04-26T21:45:57.785959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784144+0000","last_change":"2024-04-26T21:45:56.693549+0000","last_active":"2024-04-26T21:45:57.784144+0000","last_peered":"2024-04-26T21:45:57.784144+0000","last_clean":"2024-04-26T21:45:57.784144+0000","last_became_active":"2024-04-26T21:45:56.693216+0000","last_became_peered":"2024-04-26T21:45:56.693216+0000","last_unstale":"2024-04-26T21:45:57.784144+0000","last_undegraded":"2024-04-26T21:45:57.784144+0000","last_fullsized":"2024-04-26T21:45:57.784144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706610+0000","last_change":"2024-04-26T21:45:56.697472+0000","last_active":"2024-04-26T21:45:57.706610+0000","last_peered":"2024-04-26T21:45:57.706610+0000","last_clean":"2024-04-26T21:45:57.706610+0000","last_became_active":"2024-04-26T21:45:56.697216+0000","last_became_peered":"2024-04-26T21:45:56.697216+0000","last_unstale":"2024-04-26T21:45:57.706610+0000","last_undegraded":"2024-04-26T21:45:57.706610+0000","last_fullsized":"2024-04-26T21:45:57.706610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688367+0000","last_change":"2024-04-26T21:45:56.696018+0000","last_active":"2024-04-26T21:45:57.688367+0000","last_peered":"2024-04-26T21:45:57.688367+0000","last_clean":"2024-04-26T21:45:57.688367+0000","last_became_active":"2024-04-26T21:45:56.695689+0000","last_became_peered":"2024-04-26T21:45:56.695689+0000","last_unstale":"2024-04-26T21:45:57.688367+0000","last_undegraded":"2024-04-26T21:45:57.688367+0000","last_fullsized":"2024-04-26T21:45:57.688367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784032+0000","last_change":"2024-04-26T21:45:56.694683+0000","last_active":"2024-04-26T21:45:57.784032+0000","last_peered":"2024-04-26T21:45:57.784032+0000","last_clean":"2024-04-26T21:45:57.784032+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.784032+0000","last_undegraded":"2024-04-26T21:45:57.784032+0000","last_fullsized":"2024-04-26T21:45:57.784032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784298+0000","last_change":"2024-04-26T21:45:56.708141+0000","last_active":"2024-04-26T21:45:57.784298+0000","last_peered":"2024-04-26T21:45:57.784298+0000","last_clean":"2024-04-26T21:45:57.784298+0000","last_became_active":"2024-04-26T21:45:56.705466+0000","last_became_peered":"2024-04-26T21:45:56.705466+0000","last_unstale":"2024-04-26T21:45:57.784298+0000","last_undegraded":"2024-04-26T21:45:57.784298+0000","last_fullsized":"2024-04-26T21:45:57.784298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786657+0000","last_change":"2024-04-26T21:45:56.719370+0000","last_active":"2024-04-26T21:45:57.786657+0000","last_peered":"2024-04-26T21:45:57.786657+0000","last_clean":"2024-04-26T21:45:57.786657+0000","last_became_active":"2024-04-26T21:45:56.718686+0000","last_became_peered":"2024-04-26T21:45:56.718686+0000","last_unstale":"2024-04-26T21:45:57.786657+0000","last_undegraded":"2024-04-26T21:45:57.786657+0000","last_fullsized":"2024-04-26T21:45:57.786657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782446+0000","last_change":"2024-04-26T21:45:56.716691+0000","last_active":"2024-04-26T21:45:57.782446+0000","last_peered":"2024-04-26T21:45:57.782446+0000","last_clean":"2024-04-26T21:45:57.782446+0000","last_became_active":"2024-04-26T21:45:56.716457+0000","last_became_peered":"2024-04-26T21:45:56.716457+0000","last_unstale":"2024-04-26T21:45:57.782446+0000","last_undegraded":"2024-04-26T21:45:57.782446+0000","last_fullsized":"2024-04-26T21:45:57.782446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706078+0000","last_change":"2024-04-26T21:45:56.693796+0000","last_active":"2024-04-26T21:45:57.706078+0000","last_peered":"2024-04-26T21:45:57.706078+0000","last_clean":"2024-04-26T21:45:57.706078+0000","last_became_active":"2024-04-26T21:45:56.693393+0000","last_became_peered":"2024-04-26T21:45:56.693393+0000","last_unstale":"2024-04-26T21:45:57.706078+0000","last_undegraded":"2024-04-26T21:45:57.706078+0000","last_fullsized":"2024-04-26T21:45:57.706078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693646+0000","last_change":"2024-04-26T21:45:56.684700+0000","last_active":"2024-04-26T21:45:57.693646+0000","last_peered":"2024-04-26T21:45:57.693646+0000","last_clean":"2024-04-26T21:45:57.693646+0000","last_became_active":"2024-04-26T21:45:56.683789+0000","last_became_peered":"2024-04-26T21:45:56.683789+0000","last_unstale":"2024-04-26T21:45:57.693646+0000","last_undegraded":"2024-04-26T21:45:57.693646+0000","last_fullsized":"2024-04-26T21:45:57.693646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:48.558636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783980+0000","last_change":"2024-04-26T21:45:55.672715+0000","last_active":"2024-04-26T21:45:57.783980+0000","last_peered":"2024-04-26T21:45:57.783980+0000","last_clean":"2024-04-26T21:45:57.783980+0000","last_became_active":"2024-04-26T21:45:55.672430+0000","last_became_peered":"2024-04-26T21:45:55.672430+0000","last_unstale":"2024-04-26T21:45:57.783980+0000","last_undegraded":"2024-04-26T21:45:57.783980+0000","last_fullsized":"2024-04-26T21:45:57.783980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784365+0000","last_change":"2024-04-26T21:45:56.697463+0000","last_active":"2024-04-26T21:45:57.784365+0000","last_peered":"2024-04-26T21:45:57.784365+0000","last_clean":"2024-04-26T21:45:57.784365+0000","last_became_active":"2024-04-26T21:45:56.697124+0000","last_became_peered":"2024-04-26T21:45:56.697124+0000","last_unstale":"2024-04-26T21:45:57.784365+0000","last_undegraded":"2024-04-26T21:45:57.784365+0000","last_fullsized":"2024-04-26T21:45:57.784365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705510+0000","last_change":"2024-04-26T21:45:55.678579+0000","last_active":"2024-04-26T21:45:57.705510+0000","last_peered":"2024-04-26T21:45:57.705510+0000","last_clean":"2024-04-26T21:45:57.705510+0000","last_became_active":"2024-04-26T21:45:55.678193+0000","last_became_peered":"2024-04-26T21:45:55.678193+0000","last_unstale":"2024-04-26T21:45:57.705510+0000","last_undegraded":"2024-04-26T21:45:57.705510+0000","last_fullsized":"2024-04-26T21:45:57.705510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783216+0000","last_change":"2024-04-26T21:45:55.675408+0000","last_active":"2024-04-26T21:45:57.783216+0000","last_peered":"2024-04-26T21:45:57.783216+0000","last_clean":"2024-04-26T21:45:57.783216+0000","last_became_active":"2024-04-26T21:45:55.675074+0000","last_became_peered":"2024-04-26T21:45:55.675074+0000","last_unstale":"2024-04-26T21:45:57.783216+0000","last_undegraded":"2024-04-26T21:45:57.783216+0000","last_fullsized":"2024-04-26T21:45:57.783216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783259+0000","last_change":"2024-04-26T21:45:56.694914+0000","last_active":"2024-04-26T21:45:57.783259+0000","last_peered":"2024-04-26T21:45:57.783259+0000","last_clean":"2024-04-26T21:45:57.783259+0000","last_became_active":"2024-04-26T21:45:56.694634+0000","last_became_peered":"2024-04-26T21:45:56.694634+0000","last_unstale":"2024-04-26T21:45:57.783259+0000","last_undegraded":"2024-04-26T21:45:57.783259+0000","last_fullsized":"2024-04-26T21:45:57.783259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785786+0000","last_change":"2024-04-26T21:45:55.669307+0000","last_active":"2024-04-26T21:45:57.785786+0000","last_peered":"2024-04-26T21:45:57.785786+0000","last_clean":"2024-04-26T21:45:57.785786+0000","last_became_active":"2024-04-26T21:45:55.668979+0000","last_became_peered":"2024-04-26T21:45:55.668979+0000","last_unstale":"2024-04-26T21:45:57.785786+0000","last_undegraded":"2024-04-26T21:45:57.785786+0000","last_fullsized":"2024-04-26T21:45:57.785786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687926+0000","last_change":"2024-04-26T21:45:56.697521+0000","last_active":"2024-04-26T21:45:57.687926+0000","last_peered":"2024-04-26T21:45:57.687926+0000","last_clean":"2024-04-26T21:45:57.687926+0000","last_became_active":"2024-04-26T21:45:56.697165+0000","last_became_peered":"2024-04-26T21:45:56.697165+0000","last_unstale":"2024-04-26T21:45:57.687926+0000","last_undegraded":"2024-04-26T21:45:57.687926+0000","last_fullsized":"2024-04-26T21:45:57.687926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706395+0000","last_change":"2024-04-26T21:45:55.686208+0000","last_active":"2024-04-26T21:45:57.706395+0000","last_peered":"2024-04-26T21:45:57.706395+0000","last_clean":"2024-04-26T21:45:57.706395+0000","last_became_active":"2024-04-26T21:45:55.685947+0000","last_became_peered":"2024-04-26T21:45:55.685947+0000","last_unstale":"2024-04-26T21:45:57.706395+0000","last_undegraded":"2024-04-26T21:45:57.706395+0000","last_fullsized":"2024-04-26T21:45:57.706395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786494+0000","last_change":"2024-04-26T21:45:56.714928+0000","last_active":"2024-04-26T21:45:57.786494+0000","last_peered":"2024-04-26T21:45:57.786494+0000","last_clean":"2024-04-26T21:45:57.786494+0000","last_became_active":"2024-04-26T21:45:56.714595+0000","last_became_peered":"2024-04-26T21:45:56.714595+0000","last_unstale":"2024-04-26T21:45:57.786494+0000","last_undegraded":"2024-04-26T21:45:57.786494+0000","last_fullsized":"2024-04-26T21:45:57.786494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786194+0000","last_change":"2024-04-26T21:45:55.684279+0000","last_active":"2024-04-26T21:45:57.786194+0000","last_peered":"2024-04-26T21:45:57.786194+0000","last_clean":"2024-04-26T21:45:57.786194+0000","last_became_active":"2024-04-26T21:45:55.683712+0000","last_became_peered":"2024-04-26T21:45:55.683712+0000","last_unstale":"2024-04-26T21:45:57.786194+0000","last_undegraded":"2024-04-26T21:45:57.786194+0000","last_fullsized":"2024-04-26T21:45:57.786194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784002+0000","last_change":"2024-04-26T21:45:56.719558+0000","last_active":"2024-04-26T21:45:57.784002+0000","last_peered":"2024-04-26T21:45:57.784002+0000","last_clean":"2024-04-26T21:45:57.784002+0000","last_became_active":"2024-04-26T21:45:56.718765+0000","last_became_peered":"2024-04-26T21:45:56.718765+0000","last_unstale":"2024-04-26T21:45:57.784002+0000","last_undegraded":"2024-04-26T21:45:57.784002+0000","last_fullsized":"2024-04-26T21:45:57.784002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688698+0000","last_change":"2024-04-26T21:45:55.682896+0000","last_active":"2024-04-26T21:45:57.688698+0000","last_peered":"2024-04-26T21:45:57.688698+0000","last_clean":"2024-04-26T21:45:57.688698+0000","last_became_active":"2024-04-26T21:45:55.682576+0000","last_became_peered":"2024-04-26T21:45:55.682576+0000","last_unstale":"2024-04-26T21:45:57.688698+0000","last_undegraded":"2024-04-26T21:45:57.688698+0000","last_fullsized":"2024-04-26T21:45:57.688698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694424+0000","last_change":"2024-04-26T21:45:56.702692+0000","last_active":"2024-04-26T21:45:57.694424+0000","last_peered":"2024-04-26T21:45:57.694424+0000","last_clean":"2024-04-26T21:45:57.694424+0000","last_became_active":"2024-04-26T21:45:56.702369+0000","last_became_peered":"2024-04-26T21:45:56.702369+0000","last_unstale":"2024-04-26T21:45:57.694424+0000","last_undegraded":"2024-04-26T21:45:57.694424+0000","last_fullsized":"2024-04-26T21:45:57.694424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705841+0000","last_change":"2024-04-26T21:45:55.678223+0000","last_active":"2024-04-26T21:45:57.705841+0000","last_peered":"2024-04-26T21:45:57.705841+0000","last_clean":"2024-04-26T21:45:57.705841+0000","last_became_active":"2024-04-26T21:45:55.676989+0000","last_became_peered":"2024-04-26T21:45:55.676989+0000","last_unstale":"2024-04-26T21:45:57.705841+0000","last_undegraded":"2024-04-26T21:45:57.705841+0000","last_fullsized":"2024-04-26T21:45:57.705841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697495+0000","last_change":"2024-04-26T21:45:56.700587+0000","last_active":"2024-04-26T21:45:57.697495+0000","last_peered":"2024-04-26T21:45:57.697495+0000","last_clean":"2024-04-26T21:45:57.697495+0000","last_became_active":"2024-04-26T21:45:56.695698+0000","last_became_peered":"2024-04-26T21:45:56.695698+0000","last_unstale":"2024-04-26T21:45:57.697495+0000","last_undegraded":"2024-04-26T21:45:57.697495+0000","last_fullsized":"2024-04-26T21:45:57.697495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784401+0000","last_change":"2024-04-26T21:45:55.674764+0000","last_active":"2024-04-26T21:45:57.784401+0000","last_peered":"2024-04-26T21:45:57.784401+0000","last_clean":"2024-04-26T21:45:57.784401+0000","last_became_active":"2024-04-26T21:45:55.674470+0000","last_became_peered":"2024-04-26T21:45:55.674470+0000","last_unstale":"2024-04-26T21:45:57.784401+0000","last_undegraded":"2024-04-26T21:45:57.784401+0000","last_fullsized":"2024-04-26T21:45:57.784401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783659+0000","last_change":"2024-04-26T21:45:56.702112+0000","last_active":"2024-04-26T21:45:57.783659+0000","last_peered":"2024-04-26T21:45:57.783659+0000","last_clean":"2024-04-26T21:45:57.783659+0000","last_became_active":"2024-04-26T21:45:56.701847+0000","last_became_peered":"2024-04-26T21:45:56.701847+0000","last_unstale":"2024-04-26T21:45:57.783659+0000","last_undegraded":"2024-04-26T21:45:57.783659+0000","last_fullsized":"2024-04-26T21:45:57.783659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786937+0000","last_change":"2024-04-26T21:45:55.674485+0000","last_active":"2024-04-26T21:45:57.786937+0000","last_peered":"2024-04-26T21:45:57.786937+0000","last_clean":"2024-04-26T21:45:57.786937+0000","last_became_active":"2024-04-26T21:45:55.674236+0000","last_became_peered":"2024-04-26T21:45:55.674236+0000","last_unstale":"2024-04-26T21:45:57.786937+0000","last_undegraded":"2024-04-26T21:45:57.786937+0000","last_fullsized":"2024-04-26T21:45:57.786937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697306+0000","last_change":"2024-04-26T21:45:56.691631+0000","last_active":"2024-04-26T21:45:57.697306+0000","last_peered":"2024-04-26T21:45:57.697306+0000","last_clean":"2024-04-26T21:45:57.697306+0000","last_became_active":"2024-04-26T21:45:56.691262+0000","last_became_peered":"2024-04-26T21:45:56.691262+0000","last_unstale":"2024-04-26T21:45:57.697306+0000","last_undegraded":"2024-04-26T21:45:57.697306+0000","last_fullsized":"2024-04-26T21:45:57.697306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705443+0000","last_change":"2024-04-26T21:45:55.679236+0000","last_active":"2024-04-26T21:45:57.705443+0000","last_peered":"2024-04-26T21:45:57.705443+0000","last_clean":"2024-04-26T21:45:57.705443+0000","last_became_active":"2024-04-26T21:45:55.678916+0000","last_became_peered":"2024-04-26T21:45:55.678916+0000","last_unstale":"2024-04-26T21:45:57.705443+0000","last_undegraded":"2024-04-26T21:45:57.705443+0000","last_fullsized":"2024-04-26T21:45:57.705443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705838+0000","last_change":"2024-04-26T21:45:56.693186+0000","last_active":"2024-04-26T21:45:57.705838+0000","last_peered":"2024-04-26T21:45:57.705838+0000","last_clean":"2024-04-26T21:45:57.705838+0000","last_became_active":"2024-04-26T21:45:56.692617+0000","last_became_peered":"2024-04-26T21:45:56.692617+0000","last_unstale":"2024-04-26T21:45:57.705838+0000","last_undegraded":"2024-04-26T21:45:57.705838+0000","last_fullsized":"2024-04-26T21:45:57.705838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782566+0000","last_change":"2024-04-26T21:45:55.669857+0000","last_active":"2024-04-26T21:45:57.782566+0000","last_peered":"2024-04-26T21:45:57.782566+0000","last_clean":"2024-04-26T21:45:57.782566+0000","last_became_active":"2024-04-26T21:45:55.667192+0000","last_became_peered":"2024-04-26T21:45:55.667192+0000","last_unstale":"2024-04-26T21:45:57.782566+0000","last_undegraded":"2024-04-26T21:45:57.782566+0000","last_fullsized":"2024-04-26T21:45:57.782566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696941+0000","last_change":"2024-04-26T21:45:56.688026+0000","last_active":"2024-04-26T21:45:57.696941+0000","last_peered":"2024-04-26T21:45:57.696941+0000","last_clean":"2024-04-26T21:45:57.696941+0000","last_became_active":"2024-04-26T21:45:56.687620+0000","last_became_peered":"2024-04-26T21:45:56.687620+0000","last_unstale":"2024-04-26T21:45:57.696941+0000","last_undegraded":"2024-04-26T21:45:57.696941+0000","last_fullsized":"2024-04-26T21:45:57.696941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694496+0000","last_change":"2024-04-26T21:45:55.671925+0000","last_active":"2024-04-26T21:45:57.694496+0000","last_peered":"2024-04-26T21:45:57.694496+0000","last_clean":"2024-04-26T21:45:57.694496+0000","last_became_active":"2024-04-26T21:45:55.671562+0000","last_became_peered":"2024-04-26T21:45:55.671562+0000","last_unstale":"2024-04-26T21:45:57.694496+0000","last_undegraded":"2024-04-26T21:45:57.694496+0000","last_fullsized":"2024-04-26T21:45:57.694496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:47.590826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697702+0000","last_change":"2024-04-26T21:45:56.696591+0000","last_active":"2024-04-26T21:45:57.697702+0000","last_peered":"2024-04-26T21:45:57.697702+0000","last_clean":"2024-04-26T21:45:57.697702+0000","last_became_active":"2024-04-26T21:45:56.695741+0000","last_became_peered":"2024-04-26T21:45:56.695741+0000","last_unstale":"2024-04-26T21:45:57.697702+0000","last_undegraded":"2024-04-26T21:45:57.697702+0000","last_fullsized":"2024-04-26T21:45:57.697702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782611+0000","last_change":"2024-04-26T21:45:55.661721+0000","last_active":"2024-04-26T21:45:57.782611+0000","last_peered":"2024-04-26T21:45:57.782611+0000","last_clean":"2024-04-26T21:45:57.782611+0000","last_became_active":"2024-04-26T21:45:55.661317+0000","last_became_peered":"2024-04-26T21:45:55.661317+0000","last_unstale":"2024-04-26T21:45:57.782611+0000","last_undegraded":"2024-04-26T21:45:57.782611+0000","last_fullsized":"2024-04-26T21:45:57.782611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698417+0000","last_change":"2024-04-26T21:45:56.690727+0000","last_active":"2024-04-26T21:45:57.698417+0000","last_peered":"2024-04-26T21:45:57.698417+0000","last_clean":"2024-04-26T21:45:57.698417+0000","last_became_active":"2024-04-26T21:45:56.689340+0000","last_became_peered":"2024-04-26T21:45:56.689340+0000","last_unstale":"2024-04-26T21:45:57.698417+0000","last_undegraded":"2024-04-26T21:45:57.698417+0000","last_fullsized":"2024-04-26T21:45:57.698417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:16.382523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688598+0000","last_change":"2024-04-26T21:45:55.667840+0000","last_active":"2024-04-26T21:45:57.688598+0000","last_peered":"2024-04-26T21:45:57.688598+0000","last_clean":"2024-04-26T21:45:57.688598+0000","last_became_active":"2024-04-26T21:45:55.667095+0000","last_became_peered":"2024-04-26T21:45:55.667095+0000","last_unstale":"2024-04-26T21:45:57.688598+0000","last_undegraded":"2024-04-26T21:45:57.688598+0000","last_fullsized":"2024-04-26T21:45:57.688598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786997+0000","last_change":"2024-04-26T21:45:56.724926+0000","last_active":"2024-04-26T21:45:57.786997+0000","last_peered":"2024-04-26T21:45:57.786997+0000","last_clean":"2024-04-26T21:45:57.786997+0000","last_became_active":"2024-04-26T21:45:56.723585+0000","last_became_peered":"2024-04-26T21:45:56.723585+0000","last_unstale":"2024-04-26T21:45:57.786997+0000","last_undegraded":"2024-04-26T21:45:57.786997+0000","last_fullsized":"2024-04-26T21:45:57.786997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782488+0000","last_change":"2024-04-26T21:45:55.679971+0000","last_active":"2024-04-26T21:45:57.782488+0000","last_peered":"2024-04-26T21:45:57.782488+0000","last_clean":"2024-04-26T21:45:57.782488+0000","last_became_active":"2024-04-26T21:45:55.679650+0000","last_became_peered":"2024-04-26T21:45:55.679650+0000","last_unstale":"2024-04-26T21:45:57.782488+0000","last_undegraded":"2024-04-26T21:45:57.782488+0000","last_fullsized":"2024-04-26T21:45:57.782488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697335+0000","last_change":"2024-04-26T21:45:56.707369+0000","last_active":"2024-04-26T21:45:57.697335+0000","last_peered":"2024-04-26T21:45:57.697335+0000","last_clean":"2024-04-26T21:45:57.697335+0000","last_became_active":"2024-04-26T21:45:56.707148+0000","last_became_peered":"2024-04-26T21:45:56.707148+0000","last_unstale":"2024-04-26T21:45:57.697335+0000","last_undegraded":"2024-04-26T21:45:57.697335+0000","last_fullsized":"2024-04-26T21:45:57.697335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696696+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:45:57.696696+0000","last_peered":"2024-04-26T21:45:57.696696+0000","last_clean":"2024-04-26T21:45:57.696696+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:45:57.696696+0000","last_undegraded":"2024-04-26T21:45:57.696696+0000","last_fullsized":"2024-04-26T21:45:57.696696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785566+0000","last_change":"2024-04-26T21:45:55.671235+0000","last_active":"2024-04-26T21:45:57.785566+0000","last_peered":"2024-04-26T21:45:57.785566+0000","last_clean":"2024-04-26T21:45:57.785566+0000","last_became_active":"2024-04-26T21:45:55.670854+0000","last_became_peered":"2024-04-26T21:45:55.670854+0000","last_unstale":"2024-04-26T21:45:57.785566+0000","last_undegraded":"2024-04-26T21:45:57.785566+0000","last_fullsized":"2024-04-26T21:45:57.785566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783861+0000","last_change":"2024-04-26T21:45:56.723841+0000","last_active":"2024-04-26T21:45:57.783861+0000","last_peered":"2024-04-26T21:45:57.783861+0000","last_clean":"2024-04-26T21:45:57.783861+0000","last_became_active":"2024-04-26T21:45:56.723301+0000","last_became_peered":"2024-04-26T21:45:56.723301+0000","last_unstale":"2024-04-26T21:45:57.783861+0000","last_undegraded":"2024-04-26T21:45:57.783861+0000","last_fullsized":"2024-04-26T21:45:57.783861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696426+0000","last_change":"2024-04-26T21:45:55.669460+0000","last_active":"2024-04-26T21:45:57.696426+0000","last_peered":"2024-04-26T21:45:57.696426+0000","last_clean":"2024-04-26T21:45:57.696426+0000","last_became_active":"2024-04-26T21:45:55.668390+0000","last_became_peered":"2024-04-26T21:45:55.668390+0000","last_unstale":"2024-04-26T21:45:57.696426+0000","last_undegraded":"2024-04-26T21:45:57.696426+0000","last_fullsized":"2024-04-26T21:45:57.696426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783571+0000","last_change":"2024-04-26T21:45:56.700088+0000","last_active":"2024-04-26T21:45:57.783571+0000","last_peered":"2024-04-26T21:45:57.783571+0000","last_clean":"2024-04-26T21:45:57.783571+0000","last_became_active":"2024-04-26T21:45:56.699757+0000","last_became_peered":"2024-04-26T21:45:56.699757+0000","last_unstale":"2024-04-26T21:45:57.783571+0000","last_undegraded":"2024-04-26T21:45:57.783571+0000","last_fullsized":"2024-04-26T21:45:57.783571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:57.821808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782410+0000","last_change":"2024-04-26T21:45:55.662154+0000","last_active":"2024-04-26T21:45:57.782410+0000","last_peered":"2024-04-26T21:45:57.782410+0000","last_clean":"2024-04-26T21:45:57.782410+0000","last_became_active":"2024-04-26T21:45:55.661829+0000","last_became_peered":"2024-04-26T21:45:55.661829+0000","last_unstale":"2024-04-26T21:45:57.782410+0000","last_undegraded":"2024-04-26T21:45:57.782410+0000","last_fullsized":"2024-04-26T21:45:57.782410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688590+0000","last_change":"2024-04-26T21:45:56.689249+0000","last_active":"2024-04-26T21:45:57.688590+0000","last_peered":"2024-04-26T21:45:57.688590+0000","last_clean":"2024-04-26T21:45:57.688590+0000","last_became_active":"2024-04-26T21:45:56.688918+0000","last_became_peered":"2024-04-26T21:45:56.688918+0000","last_unstale":"2024-04-26T21:45:57.688590+0000","last_undegraded":"2024-04-26T21:45:57.688590+0000","last_fullsized":"2024-04-26T21:45:57.688590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786809+0000","last_change":"2024-04-26T21:45:55.672652+0000","last_active":"2024-04-26T21:45:57.786809+0000","last_peered":"2024-04-26T21:45:57.786809+0000","last_clean":"2024-04-26T21:45:57.786809+0000","last_became_active":"2024-04-26T21:45:55.672091+0000","last_became_peered":"2024-04-26T21:45:55.672091+0000","last_unstale":"2024-04-26T21:45:57.786809+0000","last_undegraded":"2024-04-26T21:45:57.786809+0000","last_fullsized":"2024-04-26T21:45:57.786809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:12.199574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693856+0000","last_change":"2024-04-26T21:45:56.697429+0000","last_active":"2024-04-26T21:45:57.693856+0000","last_peered":"2024-04-26T21:45:57.693856+0000","last_clean":"2024-04-26T21:45:57.693856+0000","last_became_active":"2024-04-26T21:45:56.697080+0000","last_became_peered":"2024-04-26T21:45:56.697080+0000","last_unstale":"2024-04-26T21:45:57.693856+0000","last_undegraded":"2024-04-26T21:45:57.693856+0000","last_fullsized":"2024-04-26T21:45:57.693856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:50.881602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687872+0000","last_change":"2024-04-26T21:45:55.667311+0000","last_active":"2024-04-26T21:45:57.687872+0000","last_peered":"2024-04-26T21:45:57.687872+0000","last_clean":"2024-04-26T21:45:57.687872+0000","last_became_active":"2024-04-26T21:45:55.667025+0000","last_became_peered":"2024-04-26T21:45:55.667025+0000","last_unstale":"2024-04-26T21:45:57.687872+0000","last_undegraded":"2024-04-26T21:45:57.687872+0000","last_fullsized":"2024-04-26T21:45:57.687872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786703+0000","last_change":"2024-04-26T21:45:56.714921+0000","last_active":"2024-04-26T21:45:57.786703+0000","last_peered":"2024-04-26T21:45:57.786703+0000","last_clean":"2024-04-26T21:45:57.786703+0000","last_became_active":"2024-04-26T21:45:56.714330+0000","last_became_peered":"2024-04-26T21:45:56.714330+0000","last_unstale":"2024-04-26T21:45:57.786703+0000","last_undegraded":"2024-04-26T21:45:57.786703+0000","last_fullsized":"2024-04-26T21:45:57.786703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697756+0000","last_change":"2024-04-26T21:45:55.673285+0000","last_active":"2024-04-26T21:45:57.697756+0000","last_peered":"2024-04-26T21:45:57.697756+0000","last_clean":"2024-04-26T21:45:57.697756+0000","last_became_active":"2024-04-26T21:45:55.672987+0000","last_became_peered":"2024-04-26T21:45:55.672987+0000","last_unstale":"2024-04-26T21:45:57.697756+0000","last_undegraded":"2024-04-26T21:45:57.697756+0000","last_fullsized":"2024-04-26T21:45:57.697756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:30.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687819+0000","last_change":"2024-04-26T21:45:56.696370+0000","last_active":"2024-04-26T21:45:57.687819+0000","last_peered":"2024-04-26T21:45:57.687819+0000","last_clean":"2024-04-26T21:45:57.687819+0000","last_became_active":"2024-04-26T21:45:56.696073+0000","last_became_peered":"2024-04-26T21:45:56.696073+0000","last_unstale":"2024-04-26T21:45:57.687819+0000","last_undegraded":"2024-04-26T21:45:57.687819+0000","last_fullsized":"2024-04-26T21:45:57.687819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688658+0000","last_change":"2024-04-26T21:45:55.683832+0000","last_active":"2024-04-26T21:45:57.688658+0000","last_peered":"2024-04-26T21:45:57.688658+0000","last_clean":"2024-04-26T21:45:57.688658+0000","last_became_active":"2024-04-26T21:45:55.683577+0000","last_became_peered":"2024-04-26T21:45:55.683577+0000","last_unstale":"2024-04-26T21:45:57.688658+0000","last_undegraded":"2024-04-26T21:45:57.688658+0000","last_fullsized":"2024-04-26T21:45:57.688658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784223+0000","last_change":"2024-04-26T21:45:56.695282+0000","last_active":"2024-04-26T21:45:57.784223+0000","last_peered":"2024-04-26T21:45:57.784223+0000","last_clean":"2024-04-26T21:45:57.784223+0000","last_became_active":"2024-04-26T21:45:56.694899+0000","last_became_peered":"2024-04-26T21:45:56.694899+0000","last_unstale":"2024-04-26T21:45:57.784223+0000","last_undegraded":"2024-04-26T21:45:57.784223+0000","last_fullsized":"2024-04-26T21:45:57.784223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706431+0000","last_change":"2024-04-26T21:45:55.684115+0000","last_active":"2024-04-26T21:45:57.706431+0000","last_peered":"2024-04-26T21:45:57.706431+0000","last_clean":"2024-04-26T21:45:57.706431+0000","last_became_active":"2024-04-26T21:45:55.683742+0000","last_became_peered":"2024-04-26T21:45:55.683742+0000","last_unstale":"2024-04-26T21:45:57.706431+0000","last_undegraded":"2024-04-26T21:45:57.706431+0000","last_fullsized":"2024-04-26T21:45:57.706431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783867+0000","last_change":"2024-04-26T21:45:56.710633+0000","last_active":"2024-04-26T21:45:57.783867+0000","last_peered":"2024-04-26T21:45:57.783867+0000","last_clean":"2024-04-26T21:45:57.783867+0000","last_became_active":"2024-04-26T21:45:56.710300+0000","last_became_peered":"2024-04-26T21:45:56.710300+0000","last_unstale":"2024-04-26T21:45:57.783867+0000","last_undegraded":"2024-04-26T21:45:57.783867+0000","last_fullsized":"2024-04-26T21:45:57.783867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696388+0000","last_change":"2024-04-26T21:45:55.669182+0000","last_active":"2024-04-26T21:45:57.696388+0000","last_peered":"2024-04-26T21:45:57.696388+0000","last_clean":"2024-04-26T21:45:57.696388+0000","last_became_active":"2024-04-26T21:45:55.668065+0000","last_became_peered":"2024-04-26T21:45:55.668065+0000","last_unstale":"2024-04-26T21:45:57.696388+0000","last_undegraded":"2024-04-26T21:45:57.696388+0000","last_fullsized":"2024-04-26T21:45:57.696388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693706+0000","last_change":"2024-04-26T21:45:56.690785+0000","last_active":"2024-04-26T21:45:57.693706+0000","last_peered":"2024-04-26T21:45:57.693706+0000","last_clean":"2024-04-26T21:45:57.693706+0000","last_became_active":"2024-04-26T21:45:56.690430+0000","last_became_peered":"2024-04-26T21:45:56.690430+0000","last_unstale":"2024-04-26T21:45:57.693706+0000","last_undegraded":"2024-04-26T21:45:57.693706+0000","last_fullsized":"2024-04-26T21:45:57.693706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:04.561079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687976+0000","last_change":"2024-04-26T21:45:55.668369+0000","last_active":"2024-04-26T21:45:57.687976+0000","last_peered":"2024-04-26T21:45:57.687976+0000","last_clean":"2024-04-26T21:45:57.687976+0000","last_became_active":"2024-04-26T21:45:55.667944+0000","last_became_peered":"2024-04-26T21:45:55.667944+0000","last_unstale":"2024-04-26T21:45:57.687976+0000","last_undegraded":"2024-04-26T21:45:57.687976+0000","last_fullsized":"2024-04-26T21:45:57.687976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783372+0000","last_change":"2024-04-26T21:45:56.703067+0000","last_active":"2024-04-26T21:45:57.783372+0000","last_peered":"2024-04-26T21:45:57.783372+0000","last_clean":"2024-04-26T21:45:57.783372+0000","last_became_active":"2024-04-26T21:45:56.702744+0000","last_became_peered":"2024-04-26T21:45:56.702744+0000","last_unstale":"2024-04-26T21:45:57.783372+0000","last_undegraded":"2024-04-26T21:45:57.783372+0000","last_fullsized":"2024-04-26T21:45:57.783372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688080+0000","last_change":"2024-04-26T21:45:55.669865+0000","last_active":"2024-04-26T21:45:57.688080+0000","last_peered":"2024-04-26T21:45:57.688080+0000","last_clean":"2024-04-26T21:45:57.688080+0000","last_became_active":"2024-04-26T21:45:55.669595+0000","last_became_peered":"2024-04-26T21:45:55.669595+0000","last_unstale":"2024-04-26T21:45:57.688080+0000","last_undegraded":"2024-04-26T21:45:57.688080+0000","last_fullsized":"2024-04-26T21:45:57.688080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783922+0000","last_change":"2024-04-26T21:45:56.699968+0000","last_active":"2024-04-26T21:45:57.783922+0000","last_peered":"2024-04-26T21:45:57.783922+0000","last_clean":"2024-04-26T21:45:57.783922+0000","last_became_active":"2024-04-26T21:45:56.699639+0000","last_became_peered":"2024-04-26T21:45:56.699639+0000","last_unstale":"2024-04-26T21:45:57.783922+0000","last_undegraded":"2024-04-26T21:45:57.783922+0000","last_fullsized":"2024-04-26T21:45:57.783922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697360+0000","last_change":"2024-04-26T21:45:55.673651+0000","last_active":"2024-04-26T21:45:57.697360+0000","last_peered":"2024-04-26T21:45:57.697360+0000","last_clean":"2024-04-26T21:45:57.697360+0000","last_became_active":"2024-04-26T21:45:55.673284+0000","last_became_peered":"2024-04-26T21:45:55.673284+0000","last_unstale":"2024-04-26T21:45:57.697360+0000","last_undegraded":"2024-04-26T21:45:57.697360+0000","last_fullsized":"2024-04-26T21:45:57.697360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705814+0000","last_change":"2024-04-26T21:45:56.694260+0000","last_active":"2024-04-26T21:45:57.705814+0000","last_peered":"2024-04-26T21:45:57.705814+0000","last_clean":"2024-04-26T21:45:57.705814+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.705814+0000","last_undegraded":"2024-04-26T21:45:57.705814+0000","last_fullsized":"2024-04-26T21:45:57.705814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786302+0000","last_change":"2024-04-26T21:45:56.715202+0000","last_active":"2024-04-26T21:45:57.786302+0000","last_peered":"2024-04-26T21:45:57.786302+0000","last_clean":"2024-04-26T21:45:57.786302+0000","last_became_active":"2024-04-26T21:45:56.714700+0000","last_became_peered":"2024-04-26T21:45:56.714700+0000","last_unstale":"2024-04-26T21:45:57.786302+0000","last_undegraded":"2024-04-26T21:45:57.786302+0000","last_fullsized":"2024-04-26T21:45:57.786302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697171+0000","last_change":"2024-04-26T21:45:55.681370+0000","last_active":"2024-04-26T21:45:57.697171+0000","last_peered":"2024-04-26T21:45:57.697171+0000","last_clean":"2024-04-26T21:45:57.697171+0000","last_became_active":"2024-04-26T21:45:55.681034+0000","last_became_peered":"2024-04-26T21:45:55.681034+0000","last_unstale":"2024-04-26T21:45:57.697171+0000","last_undegraded":"2024-04-26T21:45:57.697171+0000","last_fullsized":"2024-04-26T21:45:57.697171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786801+0000","last_change":"2024-04-26T21:45:56.713692+0000","last_active":"2024-04-26T21:45:57.786801+0000","last_peered":"2024-04-26T21:45:57.786801+0000","last_clean":"2024-04-26T21:45:57.786801+0000","last_became_active":"2024-04-26T21:45:56.713374+0000","last_became_peered":"2024-04-26T21:45:56.713374+0000","last_unstale":"2024-04-26T21:45:57.786801+0000","last_undegraded":"2024-04-26T21:45:57.786801+0000","last_fullsized":"2024-04-26T21:45:57.786801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697482+0000","last_change":"2024-04-26T21:45:55.679436+0000","last_active":"2024-04-26T21:45:57.697482+0000","last_peered":"2024-04-26T21:45:57.697482+0000","last_clean":"2024-04-26T21:45:57.697482+0000","last_became_active":"2024-04-26T21:45:55.678851+0000","last_became_peered":"2024-04-26T21:45:55.678851+0000","last_unstale":"2024-04-26T21:45:57.697482+0000","last_undegraded":"2024-04-26T21:45:57.697482+0000","last_fullsized":"2024-04-26T21:45:57.697482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:16.328469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783868+0000","last_change":"2024-04-26T21:45:56.719859+0000","last_active":"2024-04-26T21:45:57.783868+0000","last_peered":"2024-04-26T21:45:57.783868+0000","last_clean":"2024-04-26T21:45:57.783868+0000","last_became_active":"2024-04-26T21:45:56.719166+0000","last_became_peered":"2024-04-26T21:45:56.719166+0000","last_unstale":"2024-04-26T21:45:57.783868+0000","last_undegraded":"2024-04-26T21:45:57.783868+0000","last_fullsized":"2024-04-26T21:45:57.783868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784767+0000","last_change":"2024-04-26T21:45:55.664913+0000","last_active":"2024-04-26T21:45:57.784767+0000","last_peered":"2024-04-26T21:45:57.784767+0000","last_clean":"2024-04-26T21:45:57.784767+0000","last_became_active":"2024-04-26T21:45:55.664561+0000","last_became_peered":"2024-04-26T21:45:55.664561+0000","last_unstale":"2024-04-26T21:45:57.784767+0000","last_undegraded":"2024-04-26T21:45:57.784767+0000","last_fullsized":"2024-04-26T21:45:57.784767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784594+0000","last_change":"2024-04-26T21:45:56.698358+0000","last_active":"2024-04-26T21:45:57.784594+0000","last_peered":"2024-04-26T21:45:57.784594+0000","last_clean":"2024-04-26T21:45:57.784594+0000","last_became_active":"2024-04-26T21:45:56.697924+0000","last_became_peered":"2024-04-26T21:45:56.697924+0000","last_unstale":"2024-04-26T21:45:57.784594+0000","last_undegraded":"2024-04-26T21:45:57.784594+0000","last_fullsized":"2024-04-26T21:45:57.784594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783992+0000","last_change":"2024-04-26T21:45:55.668737+0000","last_active":"2024-04-26T21:45:57.783992+0000","last_peered":"2024-04-26T21:45:57.783992+0000","last_clean":"2024-04-26T21:45:57.783992+0000","last_became_active":"2024-04-26T21:45:55.668415+0000","last_became_peered":"2024-04-26T21:45:55.668415+0000","last_unstale":"2024-04-26T21:45:57.783992+0000","last_undegraded":"2024-04-26T21:45:57.783992+0000","last_fullsized":"2024-04-26T21:45:57.783992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:46:06.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:46:06.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.802+0000 7f867fe00640 1 Processor -- start 2024-04-26T21:46:06.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.802+0000 7f867fe00640 1 -- start start 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8680145570 0x7f86800d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8680073ed0 0x7f86800d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 0x7f868015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f86801606b0 con 0x7f8680073ed0 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8680073110 con 0x7f868015f940 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.803+0000 7f867fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f86800733f0 con 0x7f8680145570 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8680073ed0 0x7f86800d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8680073ed0 0x7f86800d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52332/0 (socket says 172.21.15.106:52332) 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 -- 172.21.15.106:0/2000710680 learned_addr learned my addr 172.21.15.106:0/2000710680 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f867ea00640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8680145570 0x7f86800d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f867f400640 1 --2- 172.21.15.106:0/2000710680 >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 0x7f868015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 -- 172.21.15.106:0/2000710680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8680145570 msgr2=0x7f86800d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8680145570 0x7f86800d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 -- 172.21.15.106:0/2000710680 >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 msgr2=0x7f868015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 --2- 172.21.15.106:0/2000710680 >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 0x7f868015fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f8676000640 1 -- 172.21.15.106:0/2000710680 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86800736d0 con 0x7f8680073ed0 2024-04-26T21:46:06.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.804+0000 7f867f400640 1 --2- 172.21.15.106:0/2000710680 >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 0x7f868015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:06.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.805+0000 7f8676000640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680073ed0 0x7f86800d13b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f86800d0a60 tx=0x7f866802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7eca4c942da82e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.805+0000 7f867e000640 1 -- 172.21.15.106:0/2000710680 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668037070 con 0x7f8680073ed0 2024-04-26T21:46:06.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.805+0000 7f867e000640 1 -- 172.21.15.106:0/2000710680 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668032440 con 0x7f8680073ed0 2024-04-26T21:46:06.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.805+0000 7f867e000640 1 -- 172.21.15.106:0/2000710680 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668032810 con 0x7f8680073ed0 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 -- 172.21.15.106:0/2000710680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680073ed0 msgr2=0x7f86800d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680073ed0 0x7f86800d13b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f86800d0a60 tx=0x7f866802a140 comp rx=0 tx=0).stop 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 -- 172.21.15.106:0/2000710680 shutdown_connections 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 --2- 172.21.15.106:0/2000710680 >> v2:172.21.15.204:3300/0 conn(0x7f868015f940 0x7f868015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680073ed0 0x7f86800d13b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 --2- 172.21.15.106:0/2000710680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8680145570 0x7f86800d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 -- 172.21.15.106:0/2000710680 >> 172.21.15.106:0/2000710680 conn(0x7f868006c740 msgr2=0x7f868006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 -- 172.21.15.106:0/2000710680 shutdown_connections 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.806+0000 7f867fe00640 1 -- 172.21.15.106:0/2000710680 wait complete. 2024-04-26T21:46:06.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.808+0000 7f867fe00640 1 Processor -- start 2024-04-26T21:46:06.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.808+0000 7f867fe00640 1 -- start start 2024-04-26T21:46:06.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8680073ed0 0x7f868013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868015f940 0x7f868015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86801602d0 con 0x7f8680145570 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8680145950 con 0x7f8680073ed0 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8680160450 con 0x7f868015f940 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52342/0 (socket says 172.21.15.106:52342) 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 -- 172.21.15.106:0/3631838884 learned_addr learned my addr 172.21.15.106:0/3631838884 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 -- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868015f940 msgr2=0x7f868015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868015f940 0x7f868015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f867ea00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8680073ed0 0x7f868013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 -- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8680073ed0 msgr2=0x7f868013e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8680073ed0 0x7f868013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.809+0000 7f8676000640 1 -- 172.21.15.106:0/3631838884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86800b0230 con 0x7f8680145570 2024-04-26T21:46:06.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f8676000640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8668000c00 tx=0x7f86680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668041020 con 0x7f8680145570 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668030800 con 0x7f8680145570 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86800afc70 con 0x7f8680145570 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668037040 con 0x7f8680145570 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.810+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86800ce5b0 con 0x7f8680145570 2024-04-26T21:46:06.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.811+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86800cd830 con 0x7f8680145570 2024-04-26T21:46:06.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.812+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8668044020 con 0x7f8680145570 2024-04-26T21:46:06.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.813+0000 7f8677400640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 0x7f865c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.813+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86680ac100 con 0x7f8680145570 2024-04-26T21:46:06.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.813+0000 7f867ea00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 0x7f865c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.813+0000 7f867ea00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 0x7f865c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8670006fd0 tx=0x7f8670008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:06.817+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86800cd830 con 0x7f8680145570 2024-04-26T21:46:07.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.004+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86800729e0 con 0x7f865c06c8f0 2024-04-26T21:46:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.023+0000 7f8677400640 1 -- 172.21.15.106:0/3631838884 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f86800729e0 con 0x7f865c06c8f0 2024-04-26T21:46:07.025 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:07.028 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:46:07.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 msgr2=0x7f865c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 0x7f865c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8670006fd0 tx=0x7f8670008040 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 msgr2=0x7f86800af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8668000c00 tx=0x7f86680046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 shutdown_connections 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06c8f0 0x7f865c06edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868015f940 0x7f868015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8680145570 0x7f86800af2d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 --2- 172.21.15.106:0/3631838884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8680073ed0 0x7f868013e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 >> 172.21.15.106:0/3631838884 conn(0x7f868006c740 msgr2=0x7f86800b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 shutdown_connections 2024-04-26T21:46:07.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.029+0000 7f867fe00640 1 -- 172.21.15.106:0/3631838884 wait complete. 2024-04-26T21:46:07.046 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T21:46:05.090837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.430104"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698213+0000","last_change":"2024-04-26T21:45:56.703198+0000","last_active":"2024-04-26T21:45:57.698213+0000","last_peered":"2024-04-26T21:45:57.698213+0000","last_clean":"2024-04-26T21:45:57.698213+0000","last_became_active":"2024-04-26T21:45:56.700438+0000","last_became_peered":"2024-04-26T21:45:56.700438+0000","last_unstale":"2024-04-26T21:45:57.698213+0000","last_undegraded":"2024-04-26T21:45:57.698213+0000","last_fullsized":"2024-04-26T21:45:57.698213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:44.744042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688536+0000","last_change":"2024-04-26T21:45:56.699815+0000","last_active":"2024-04-26T21:45:57.688536+0000","last_peered":"2024-04-26T21:45:57.688536+0000","last_clean":"2024-04-26T21:45:57.688536+0000","last_became_active":"2024-04-26T21:45:56.699515+0000","last_became_peered":"2024-04-26T21:45:56.699515+0000","last_unstale":"2024-04-26T21:45:57.688536+0000","last_undegraded":"2024-04-26T21:45:57.688536+0000","last_fullsized":"2024-04-26T21:45:57.688536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784161+0000","last_change":"2024-04-26T21:45:56.696080+0000","last_active":"2024-04-26T21:45:57.784161+0000","last_peered":"2024-04-26T21:45:57.784161+0000","last_clean":"2024-04-26T21:45:57.784161+0000","last_became_active":"2024-04-26T21:45:56.695767+0000","last_became_peered":"2024-04-26T21:45:56.695767+0000","last_unstale":"2024-04-26T21:45:57.784161+0000","last_undegraded":"2024-04-26T21:45:57.784161+0000","last_fullsized":"2024-04-26T21:45:57.784161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786723+0000","last_change":"2024-04-26T21:45:56.724164+0000","last_active":"2024-04-26T21:45:57.786723+0000","last_peered":"2024-04-26T21:45:57.786723+0000","last_clean":"2024-04-26T21:45:57.786723+0000","last_became_active":"2024-04-26T21:45:56.719445+0000","last_became_peered":"2024-04-26T21:45:56.719445+0000","last_unstale":"2024-04-26T21:45:57.786723+0000","last_undegraded":"2024-04-26T21:45:57.786723+0000","last_fullsized":"2024-04-26T21:45:57.786723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697154+0000","last_change":"2024-04-26T21:45:56.692330+0000","last_active":"2024-04-26T21:45:57.697154+0000","last_peered":"2024-04-26T21:45:57.697154+0000","last_clean":"2024-04-26T21:45:57.697154+0000","last_became_active":"2024-04-26T21:45:56.692002+0000","last_became_peered":"2024-04-26T21:45:56.692002+0000","last_unstale":"2024-04-26T21:45:57.697154+0000","last_undegraded":"2024-04-26T21:45:57.697154+0000","last_fullsized":"2024-04-26T21:45:57.697154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786129+0000","last_change":"2024-04-26T21:45:56.719030+0000","last_active":"2024-04-26T21:45:57.786129+0000","last_peered":"2024-04-26T21:45:57.786129+0000","last_clean":"2024-04-26T21:45:57.786129+0000","last_became_active":"2024-04-26T21:45:56.718334+0000","last_became_peered":"2024-04-26T21:45:56.718334+0000","last_unstale":"2024-04-26T21:45:57.786129+0000","last_undegraded":"2024-04-26T21:45:57.786129+0000","last_fullsized":"2024-04-26T21:45:57.786129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696797+0000","last_change":"2024-04-26T21:45:56.692710+0000","last_active":"2024-04-26T21:45:57.696797+0000","last_peered":"2024-04-26T21:45:57.696797+0000","last_clean":"2024-04-26T21:45:57.696797+0000","last_became_active":"2024-04-26T21:45:56.692374+0000","last_became_peered":"2024-04-26T21:45:56.692374+0000","last_unstale":"2024-04-26T21:45:57.696797+0000","last_undegraded":"2024-04-26T21:45:57.696797+0000","last_fullsized":"2024-04-26T21:45:57.696797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783131+0000","last_change":"2024-04-26T21:45:56.718800+0000","last_active":"2024-04-26T21:45:57.783131+0000","last_peered":"2024-04-26T21:45:57.783131+0000","last_clean":"2024-04-26T21:45:57.783131+0000","last_became_active":"2024-04-26T21:45:56.717337+0000","last_became_peered":"2024-04-26T21:45:56.717337+0000","last_unstale":"2024-04-26T21:45:57.783131+0000","last_undegraded":"2024-04-26T21:45:57.783131+0000","last_fullsized":"2024-04-26T21:45:57.783131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698512+0000","last_change":"2024-04-26T21:45:56.722478+0000","last_active":"2024-04-26T21:45:57.698512+0000","last_peered":"2024-04-26T21:45:57.698512+0000","last_clean":"2024-04-26T21:45:57.698512+0000","last_became_active":"2024-04-26T21:45:56.722182+0000","last_became_peered":"2024-04-26T21:45:56.722182+0000","last_unstale":"2024-04-26T21:45:57.698512+0000","last_undegraded":"2024-04-26T21:45:57.698512+0000","last_fullsized":"2024-04-26T21:45:57.698512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:22.652553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706561+0000","last_change":"2024-04-26T21:45:56.716505+0000","last_active":"2024-04-26T21:45:57.706561+0000","last_peered":"2024-04-26T21:45:57.706561+0000","last_clean":"2024-04-26T21:45:57.706561+0000","last_became_active":"2024-04-26T21:45:56.716200+0000","last_became_peered":"2024-04-26T21:45:56.716200+0000","last_unstale":"2024-04-26T21:45:57.706561+0000","last_undegraded":"2024-04-26T21:45:57.706561+0000","last_fullsized":"2024-04-26T21:45:57.706561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706119+0000","last_change":"2024-04-26T21:45:56.719125+0000","last_active":"2024-04-26T21:45:57.706119+0000","last_peered":"2024-04-26T21:45:57.706119+0000","last_clean":"2024-04-26T21:45:57.706119+0000","last_became_active":"2024-04-26T21:45:56.718753+0000","last_became_peered":"2024-04-26T21:45:56.718753+0000","last_unstale":"2024-04-26T21:45:57.706119+0000","last_undegraded":"2024-04-26T21:45:57.706119+0000","last_fullsized":"2024-04-26T21:45:57.706119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694414+0000","last_change":"2024-04-26T21:45:56.703498+0000","last_active":"2024-04-26T21:45:57.694414+0000","last_peered":"2024-04-26T21:45:57.694414+0000","last_clean":"2024-04-26T21:45:57.694414+0000","last_became_active":"2024-04-26T21:45:56.702682+0000","last_became_peered":"2024-04-26T21:45:56.702682+0000","last_unstale":"2024-04-26T21:45:57.694414+0000","last_undegraded":"2024-04-26T21:45:57.694414+0000","last_fullsized":"2024-04-26T21:45:57.694414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783918+0000","last_change":"2024-04-26T21:45:56.690293+0000","last_active":"2024-04-26T21:45:57.783918+0000","last_peered":"2024-04-26T21:45:57.783918+0000","last_clean":"2024-04-26T21:45:57.783918+0000","last_became_active":"2024-04-26T21:45:56.689962+0000","last_became_peered":"2024-04-26T21:45:56.689962+0000","last_unstale":"2024-04-26T21:45:57.783918+0000","last_undegraded":"2024-04-26T21:45:57.783918+0000","last_fullsized":"2024-04-26T21:45:57.783918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783643+0000","last_change":"2024-04-26T21:45:56.701473+0000","last_active":"2024-04-26T21:45:57.783643+0000","last_peered":"2024-04-26T21:45:57.783643+0000","last_clean":"2024-04-26T21:45:57.783643+0000","last_became_active":"2024-04-26T21:45:56.697858+0000","last_became_peered":"2024-04-26T21:45:56.697858+0000","last_unstale":"2024-04-26T21:45:57.783643+0000","last_undegraded":"2024-04-26T21:45:57.783643+0000","last_fullsized":"2024-04-26T21:45:57.783643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:13.204924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687846+0000","last_change":"2024-04-26T21:45:56.684598+0000","last_active":"2024-04-26T21:45:57.687846+0000","last_peered":"2024-04-26T21:45:57.687846+0000","last_clean":"2024-04-26T21:45:57.687846+0000","last_became_active":"2024-04-26T21:45:56.684065+0000","last_became_peered":"2024-04-26T21:45:56.684065+0000","last_unstale":"2024-04-26T21:45:57.687846+0000","last_undegraded":"2024-04-26T21:45:57.687846+0000","last_fullsized":"2024-04-26T21:45:57.687846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785651+0000","last_change":"2024-04-26T21:45:56.713842+0000","last_active":"2024-04-26T21:45:57.785651+0000","last_peered":"2024-04-26T21:45:57.785651+0000","last_clean":"2024-04-26T21:45:57.785651+0000","last_became_active":"2024-04-26T21:45:56.713554+0000","last_became_peered":"2024-04-26T21:45:56.713554+0000","last_unstale":"2024-04-26T21:45:57.785651+0000","last_undegraded":"2024-04-26T21:45:57.785651+0000","last_fullsized":"2024-04-26T21:45:57.785651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.787120+0000","last_change":"2024-04-26T21:45:56.717288+0000","last_active":"2024-04-26T21:45:57.787120+0000","last_peered":"2024-04-26T21:45:57.787120+0000","last_clean":"2024-04-26T21:45:57.787120+0000","last_became_active":"2024-04-26T21:45:56.716846+0000","last_became_peered":"2024-04-26T21:45:56.716846+0000","last_unstale":"2024-04-26T21:45:57.787120+0000","last_undegraded":"2024-04-26T21:45:57.787120+0000","last_fullsized":"2024-04-26T21:45:57.787120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782583+0000","last_change":"2024-04-26T21:45:56.709803+0000","last_active":"2024-04-26T21:45:57.782583+0000","last_peered":"2024-04-26T21:45:57.782583+0000","last_clean":"2024-04-26T21:45:57.782583+0000","last_became_active":"2024-04-26T21:45:56.709518+0000","last_became_peered":"2024-04-26T21:45:56.709518+0000","last_unstale":"2024-04-26T21:45:57.782583+0000","last_undegraded":"2024-04-26T21:45:57.782583+0000","last_fullsized":"2024-04-26T21:45:57.782583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697786+0000","last_change":"2024-04-26T21:45:56.691493+0000","last_active":"2024-04-26T21:45:57.697786+0000","last_peered":"2024-04-26T21:45:57.697786+0000","last_clean":"2024-04-26T21:45:57.697786+0000","last_became_active":"2024-04-26T21:45:56.691171+0000","last_became_peered":"2024-04-26T21:45:56.691171+0000","last_unstale":"2024-04-26T21:45:57.697786+0000","last_undegraded":"2024-04-26T21:45:57.697786+0000","last_fullsized":"2024-04-26T21:45:57.697786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706090+0000","last_change":"2024-04-26T21:45:56.696933+0000","last_active":"2024-04-26T21:45:57.706090+0000","last_peered":"2024-04-26T21:45:57.706090+0000","last_clean":"2024-04-26T21:45:57.706090+0000","last_became_active":"2024-04-26T21:45:56.696509+0000","last_became_peered":"2024-04-26T21:45:56.696509+0000","last_unstale":"2024-04-26T21:45:57.706090+0000","last_undegraded":"2024-04-26T21:45:57.706090+0000","last_fullsized":"2024-04-26T21:45:57.706090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784228+0000","last_change":"2024-04-26T21:45:56.696412+0000","last_active":"2024-04-26T21:45:57.784228+0000","last_peered":"2024-04-26T21:45:57.784228+0000","last_clean":"2024-04-26T21:45:57.784228+0000","last_became_active":"2024-04-26T21:45:56.695758+0000","last_became_peered":"2024-04-26T21:45:56.695758+0000","last_unstale":"2024-04-26T21:45:57.784228+0000","last_undegraded":"2024-04-26T21:45:57.784228+0000","last_fullsized":"2024-04-26T21:45:57.784228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786146+0000","last_change":"2024-04-26T21:45:56.717913+0000","last_active":"2024-04-26T21:45:57.786146+0000","last_peered":"2024-04-26T21:45:57.786146+0000","last_clean":"2024-04-26T21:45:57.786146+0000","last_became_active":"2024-04-26T21:45:56.717579+0000","last_became_peered":"2024-04-26T21:45:56.717579+0000","last_unstale":"2024-04-26T21:45:57.786146+0000","last_undegraded":"2024-04-26T21:45:57.786146+0000","last_fullsized":"2024-04-26T21:45:57.786146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783112+0000","last_change":"2024-04-26T21:45:56.694204+0000","last_active":"2024-04-26T21:45:57.783112+0000","last_peered":"2024-04-26T21:45:57.783112+0000","last_clean":"2024-04-26T21:45:57.783112+0000","last_became_active":"2024-04-26T21:45:56.693845+0000","last_became_peered":"2024-04-26T21:45:56.693845+0000","last_unstale":"2024-04-26T21:45:57.783112+0000","last_undegraded":"2024-04-26T21:45:57.783112+0000","last_fullsized":"2024-04-26T21:45:57.783112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785959+0000","last_change":"2024-04-26T21:45:56.710113+0000","last_active":"2024-04-26T21:45:57.785959+0000","last_peered":"2024-04-26T21:45:57.785959+0000","last_clean":"2024-04-26T21:45:57.785959+0000","last_became_active":"2024-04-26T21:45:56.709659+0000","last_became_peered":"2024-04-26T21:45:56.709659+0000","last_unstale":"2024-04-26T21:45:57.785959+0000","last_undegraded":"2024-04-26T21:45:57.785959+0000","last_fullsized":"2024-04-26T21:45:57.785959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784144+0000","last_change":"2024-04-26T21:45:56.693549+0000","last_active":"2024-04-26T21:45:57.784144+0000","last_peered":"2024-04-26T21:45:57.784144+0000","last_clean":"2024-04-26T21:45:57.784144+0000","last_became_active":"2024-04-26T21:45:56.693216+0000","last_became_peered":"2024-04-26T21:45:56.693216+0000","last_unstale":"2024-04-26T21:45:57.784144+0000","last_undegraded":"2024-04-26T21:45:57.784144+0000","last_fullsized":"2024-04-26T21:45:57.784144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706610+0000","last_change":"2024-04-26T21:45:56.697472+0000","last_active":"2024-04-26T21:45:57.706610+0000","last_peered":"2024-04-26T21:45:57.706610+0000","last_clean":"2024-04-26T21:45:57.706610+0000","last_became_active":"2024-04-26T21:45:56.697216+0000","last_became_peered":"2024-04-26T21:45:56.697216+0000","last_unstale":"2024-04-26T21:45:57.706610+0000","last_undegraded":"2024-04-26T21:45:57.706610+0000","last_fullsized":"2024-04-26T21:45:57.706610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688367+0000","last_change":"2024-04-26T21:45:56.696018+0000","last_active":"2024-04-26T21:45:57.688367+0000","last_peered":"2024-04-26T21:45:57.688367+0000","last_clean":"2024-04-26T21:45:57.688367+0000","last_became_active":"2024-04-26T21:45:56.695689+0000","last_became_peered":"2024-04-26T21:45:56.695689+0000","last_unstale":"2024-04-26T21:45:57.688367+0000","last_undegraded":"2024-04-26T21:45:57.688367+0000","last_fullsized":"2024-04-26T21:45:57.688367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:55.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784032+0000","last_change":"2024-04-26T21:45:56.694683+0000","last_active":"2024-04-26T21:45:57.784032+0000","last_peered":"2024-04-26T21:45:57.784032+0000","last_clean":"2024-04-26T21:45:57.784032+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.784032+0000","last_undegraded":"2024-04-26T21:45:57.784032+0000","last_fullsized":"2024-04-26T21:45:57.784032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784298+0000","last_change":"2024-04-26T21:45:56.708141+0000","last_active":"2024-04-26T21:45:57.784298+0000","last_peered":"2024-04-26T21:45:57.784298+0000","last_clean":"2024-04-26T21:45:57.784298+0000","last_became_active":"2024-04-26T21:45:56.705466+0000","last_became_peered":"2024-04-26T21:45:56.705466+0000","last_unstale":"2024-04-26T21:45:57.784298+0000","last_undegraded":"2024-04-26T21:45:57.784298+0000","last_fullsized":"2024-04-26T21:45:57.784298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786657+0000","last_change":"2024-04-26T21:45:56.719370+0000","last_active":"2024-04-26T21:45:57.786657+0000","last_peered":"2024-04-26T21:45:57.786657+0000","last_clean":"2024-04-26T21:45:57.786657+0000","last_became_active":"2024-04-26T21:45:56.718686+0000","last_became_peered":"2024-04-26T21:45:56.718686+0000","last_unstale":"2024-04-26T21:45:57.786657+0000","last_undegraded":"2024-04-26T21:45:57.786657+0000","last_fullsized":"2024-04-26T21:45:57.786657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782446+0000","last_change":"2024-04-26T21:45:56.716691+0000","last_active":"2024-04-26T21:45:57.782446+0000","last_peered":"2024-04-26T21:45:57.782446+0000","last_clean":"2024-04-26T21:45:57.782446+0000","last_became_active":"2024-04-26T21:45:56.716457+0000","last_became_peered":"2024-04-26T21:45:56.716457+0000","last_unstale":"2024-04-26T21:45:57.782446+0000","last_undegraded":"2024-04-26T21:45:57.782446+0000","last_fullsized":"2024-04-26T21:45:57.782446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:05.922909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706078+0000","last_change":"2024-04-26T21:45:56.693796+0000","last_active":"2024-04-26T21:45:57.706078+0000","last_peered":"2024-04-26T21:45:57.706078+0000","last_clean":"2024-04-26T21:45:57.706078+0000","last_became_active":"2024-04-26T21:45:56.693393+0000","last_became_peered":"2024-04-26T21:45:56.693393+0000","last_unstale":"2024-04-26T21:45:57.706078+0000","last_undegraded":"2024-04-26T21:45:57.706078+0000","last_fullsized":"2024-04-26T21:45:57.706078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693646+0000","last_change":"2024-04-26T21:45:56.684700+0000","last_active":"2024-04-26T21:45:57.693646+0000","last_peered":"2024-04-26T21:45:57.693646+0000","last_clean":"2024-04-26T21:45:57.693646+0000","last_became_active":"2024-04-26T21:45:56.683789+0000","last_became_peered":"2024-04-26T21:45:56.683789+0000","last_unstale":"2024-04-26T21:45:57.693646+0000","last_undegraded":"2024-04-26T21:45:57.693646+0000","last_fullsized":"2024-04-26T21:45:57.693646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:48.558636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783980+0000","last_change":"2024-04-26T21:45:55.672715+0000","last_active":"2024-04-26T21:45:57.783980+0000","last_peered":"2024-04-26T21:45:57.783980+0000","last_clean":"2024-04-26T21:45:57.783980+0000","last_became_active":"2024-04-26T21:45:55.672430+0000","last_became_peered":"2024-04-26T21:45:55.672430+0000","last_unstale":"2024-04-26T21:45:57.783980+0000","last_undegraded":"2024-04-26T21:45:57.783980+0000","last_fullsized":"2024-04-26T21:45:57.783980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784365+0000","last_change":"2024-04-26T21:45:56.697463+0000","last_active":"2024-04-26T21:45:57.784365+0000","last_peered":"2024-04-26T21:45:57.784365+0000","last_clean":"2024-04-26T21:45:57.784365+0000","last_became_active":"2024-04-26T21:45:56.697124+0000","last_became_peered":"2024-04-26T21:45:56.697124+0000","last_unstale":"2024-04-26T21:45:57.784365+0000","last_undegraded":"2024-04-26T21:45:57.784365+0000","last_fullsized":"2024-04-26T21:45:57.784365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705510+0000","last_change":"2024-04-26T21:45:55.678579+0000","last_active":"2024-04-26T21:45:57.705510+0000","last_peered":"2024-04-26T21:45:57.705510+0000","last_clean":"2024-04-26T21:45:57.705510+0000","last_became_active":"2024-04-26T21:45:55.678193+0000","last_became_peered":"2024-04-26T21:45:55.678193+0000","last_unstale":"2024-04-26T21:45:57.705510+0000","last_undegraded":"2024-04-26T21:45:57.705510+0000","last_fullsized":"2024-04-26T21:45:57.705510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783216+0000","last_change":"2024-04-26T21:45:55.675408+0000","last_active":"2024-04-26T21:45:57.783216+0000","last_peered":"2024-04-26T21:45:57.783216+0000","last_clean":"2024-04-26T21:45:57.783216+0000","last_became_active":"2024-04-26T21:45:55.675074+0000","last_became_peered":"2024-04-26T21:45:55.675074+0000","last_unstale":"2024-04-26T21:45:57.783216+0000","last_undegraded":"2024-04-26T21:45:57.783216+0000","last_fullsized":"2024-04-26T21:45:57.783216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783259+0000","last_change":"2024-04-26T21:45:56.694914+0000","last_active":"2024-04-26T21:45:57.783259+0000","last_peered":"2024-04-26T21:45:57.783259+0000","last_clean":"2024-04-26T21:45:57.783259+0000","last_became_active":"2024-04-26T21:45:56.694634+0000","last_became_peered":"2024-04-26T21:45:56.694634+0000","last_unstale":"2024-04-26T21:45:57.783259+0000","last_undegraded":"2024-04-26T21:45:57.783259+0000","last_fullsized":"2024-04-26T21:45:57.783259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785786+0000","last_change":"2024-04-26T21:45:55.669307+0000","last_active":"2024-04-26T21:45:57.785786+0000","last_peered":"2024-04-26T21:45:57.785786+0000","last_clean":"2024-04-26T21:45:57.785786+0000","last_became_active":"2024-04-26T21:45:55.668979+0000","last_became_peered":"2024-04-26T21:45:55.668979+0000","last_unstale":"2024-04-26T21:45:57.785786+0000","last_undegraded":"2024-04-26T21:45:57.785786+0000","last_fullsized":"2024-04-26T21:45:57.785786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687926+0000","last_change":"2024-04-26T21:45:56.697521+0000","last_active":"2024-04-26T21:45:57.687926+0000","last_peered":"2024-04-26T21:45:57.687926+0000","last_clean":"2024-04-26T21:45:57.687926+0000","last_became_active":"2024-04-26T21:45:56.697165+0000","last_became_peered":"2024-04-26T21:45:56.697165+0000","last_unstale":"2024-04-26T21:45:57.687926+0000","last_undegraded":"2024-04-26T21:45:57.687926+0000","last_fullsized":"2024-04-26T21:45:57.687926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706395+0000","last_change":"2024-04-26T21:45:55.686208+0000","last_active":"2024-04-26T21:45:57.706395+0000","last_peered":"2024-04-26T21:45:57.706395+0000","last_clean":"2024-04-26T21:45:57.706395+0000","last_became_active":"2024-04-26T21:45:55.685947+0000","last_became_peered":"2024-04-26T21:45:55.685947+0000","last_unstale":"2024-04-26T21:45:57.706395+0000","last_undegraded":"2024-04-26T21:45:57.706395+0000","last_fullsized":"2024-04-26T21:45:57.706395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786494+0000","last_change":"2024-04-26T21:45:56.714928+0000","last_active":"2024-04-26T21:45:57.786494+0000","last_peered":"2024-04-26T21:45:57.786494+0000","last_clean":"2024-04-26T21:45:57.786494+0000","last_became_active":"2024-04-26T21:45:56.714595+0000","last_became_peered":"2024-04-26T21:45:56.714595+0000","last_unstale":"2024-04-26T21:45:57.786494+0000","last_undegraded":"2024-04-26T21:45:57.786494+0000","last_fullsized":"2024-04-26T21:45:57.786494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786194+0000","last_change":"2024-04-26T21:45:55.684279+0000","last_active":"2024-04-26T21:45:57.786194+0000","last_peered":"2024-04-26T21:45:57.786194+0000","last_clean":"2024-04-26T21:45:57.786194+0000","last_became_active":"2024-04-26T21:45:55.683712+0000","last_became_peered":"2024-04-26T21:45:55.683712+0000","last_unstale":"2024-04-26T21:45:57.786194+0000","last_undegraded":"2024-04-26T21:45:57.786194+0000","last_fullsized":"2024-04-26T21:45:57.786194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784002+0000","last_change":"2024-04-26T21:45:56.719558+0000","last_active":"2024-04-26T21:45:57.784002+0000","last_peered":"2024-04-26T21:45:57.784002+0000","last_clean":"2024-04-26T21:45:57.784002+0000","last_became_active":"2024-04-26T21:45:56.718765+0000","last_became_peered":"2024-04-26T21:45:56.718765+0000","last_unstale":"2024-04-26T21:45:57.784002+0000","last_undegraded":"2024-04-26T21:45:57.784002+0000","last_fullsized":"2024-04-26T21:45:57.784002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688698+0000","last_change":"2024-04-26T21:45:55.682896+0000","last_active":"2024-04-26T21:45:57.688698+0000","last_peered":"2024-04-26T21:45:57.688698+0000","last_clean":"2024-04-26T21:45:57.688698+0000","last_became_active":"2024-04-26T21:45:55.682576+0000","last_became_peered":"2024-04-26T21:45:55.682576+0000","last_unstale":"2024-04-26T21:45:57.688698+0000","last_undegraded":"2024-04-26T21:45:57.688698+0000","last_fullsized":"2024-04-26T21:45:57.688698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694424+0000","last_change":"2024-04-26T21:45:56.702692+0000","last_active":"2024-04-26T21:45:57.694424+0000","last_peered":"2024-04-26T21:45:57.694424+0000","last_clean":"2024-04-26T21:45:57.694424+0000","last_became_active":"2024-04-26T21:45:56.702369+0000","last_became_peered":"2024-04-26T21:45:56.702369+0000","last_unstale":"2024-04-26T21:45:57.694424+0000","last_undegraded":"2024-04-26T21:45:57.694424+0000","last_fullsized":"2024-04-26T21:45:57.694424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:06.697634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705841+0000","last_change":"2024-04-26T21:45:55.678223+0000","last_active":"2024-04-26T21:45:57.705841+0000","last_peered":"2024-04-26T21:45:57.705841+0000","last_clean":"2024-04-26T21:45:57.705841+0000","last_became_active":"2024-04-26T21:45:55.676989+0000","last_became_peered":"2024-04-26T21:45:55.676989+0000","last_unstale":"2024-04-26T21:45:57.705841+0000","last_undegraded":"2024-04-26T21:45:57.705841+0000","last_fullsized":"2024-04-26T21:45:57.705841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697495+0000","last_change":"2024-04-26T21:45:56.700587+0000","last_active":"2024-04-26T21:45:57.697495+0000","last_peered":"2024-04-26T21:45:57.697495+0000","last_clean":"2024-04-26T21:45:57.697495+0000","last_became_active":"2024-04-26T21:45:56.695698+0000","last_became_peered":"2024-04-26T21:45:56.695698+0000","last_unstale":"2024-04-26T21:45:57.697495+0000","last_undegraded":"2024-04-26T21:45:57.697495+0000","last_fullsized":"2024-04-26T21:45:57.697495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784401+0000","last_change":"2024-04-26T21:45:55.674764+0000","last_active":"2024-04-26T21:45:57.784401+0000","last_peered":"2024-04-26T21:45:57.784401+0000","last_clean":"2024-04-26T21:45:57.784401+0000","last_became_active":"2024-04-26T21:45:55.674470+0000","last_became_peered":"2024-04-26T21:45:55.674470+0000","last_unstale":"2024-04-26T21:45:57.784401+0000","last_undegraded":"2024-04-26T21:45:57.784401+0000","last_fullsized":"2024-04-26T21:45:57.784401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783659+0000","last_change":"2024-04-26T21:45:56.702112+0000","last_active":"2024-04-26T21:45:57.783659+0000","last_peered":"2024-04-26T21:45:57.783659+0000","last_clean":"2024-04-26T21:45:57.783659+0000","last_became_active":"2024-04-26T21:45:56.701847+0000","last_became_peered":"2024-04-26T21:45:56.701847+0000","last_unstale":"2024-04-26T21:45:57.783659+0000","last_undegraded":"2024-04-26T21:45:57.783659+0000","last_fullsized":"2024-04-26T21:45:57.783659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786937+0000","last_change":"2024-04-26T21:45:55.674485+0000","last_active":"2024-04-26T21:45:57.786937+0000","last_peered":"2024-04-26T21:45:57.786937+0000","last_clean":"2024-04-26T21:45:57.786937+0000","last_became_active":"2024-04-26T21:45:55.674236+0000","last_became_peered":"2024-04-26T21:45:55.674236+0000","last_unstale":"2024-04-26T21:45:57.786937+0000","last_undegraded":"2024-04-26T21:45:57.786937+0000","last_fullsized":"2024-04-26T21:45:57.786937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697306+0000","last_change":"2024-04-26T21:45:56.691631+0000","last_active":"2024-04-26T21:45:57.697306+0000","last_peered":"2024-04-26T21:45:57.697306+0000","last_clean":"2024-04-26T21:45:57.697306+0000","last_became_active":"2024-04-26T21:45:56.691262+0000","last_became_peered":"2024-04-26T21:45:56.691262+0000","last_unstale":"2024-04-26T21:45:57.697306+0000","last_undegraded":"2024-04-26T21:45:57.697306+0000","last_fullsized":"2024-04-26T21:45:57.697306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705443+0000","last_change":"2024-04-26T21:45:55.679236+0000","last_active":"2024-04-26T21:45:57.705443+0000","last_peered":"2024-04-26T21:45:57.705443+0000","last_clean":"2024-04-26T21:45:57.705443+0000","last_became_active":"2024-04-26T21:45:55.678916+0000","last_became_peered":"2024-04-26T21:45:55.678916+0000","last_unstale":"2024-04-26T21:45:57.705443+0000","last_undegraded":"2024-04-26T21:45:57.705443+0000","last_fullsized":"2024-04-26T21:45:57.705443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705838+0000","last_change":"2024-04-26T21:45:56.693186+0000","last_active":"2024-04-26T21:45:57.705838+0000","last_peered":"2024-04-26T21:45:57.705838+0000","last_clean":"2024-04-26T21:45:57.705838+0000","last_became_active":"2024-04-26T21:45:56.692617+0000","last_became_peered":"2024-04-26T21:45:56.692617+0000","last_unstale":"2024-04-26T21:45:57.705838+0000","last_undegraded":"2024-04-26T21:45:57.705838+0000","last_fullsized":"2024-04-26T21:45:57.705838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782566+0000","last_change":"2024-04-26T21:45:55.669857+0000","last_active":"2024-04-26T21:45:57.782566+0000","last_peered":"2024-04-26T21:45:57.782566+0000","last_clean":"2024-04-26T21:45:57.782566+0000","last_became_active":"2024-04-26T21:45:55.667192+0000","last_became_peered":"2024-04-26T21:45:55.667192+0000","last_unstale":"2024-04-26T21:45:57.782566+0000","last_undegraded":"2024-04-26T21:45:57.782566+0000","last_fullsized":"2024-04-26T21:45:57.782566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:33.052416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696941+0000","last_change":"2024-04-26T21:45:56.688026+0000","last_active":"2024-04-26T21:45:57.696941+0000","last_peered":"2024-04-26T21:45:57.696941+0000","last_clean":"2024-04-26T21:45:57.696941+0000","last_became_active":"2024-04-26T21:45:56.687620+0000","last_became_peered":"2024-04-26T21:45:56.687620+0000","last_unstale":"2024-04-26T21:45:57.696941+0000","last_undegraded":"2024-04-26T21:45:57.696941+0000","last_fullsized":"2024-04-26T21:45:57.696941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.694496+0000","last_change":"2024-04-26T21:45:55.671925+0000","last_active":"2024-04-26T21:45:57.694496+0000","last_peered":"2024-04-26T21:45:57.694496+0000","last_clean":"2024-04-26T21:45:57.694496+0000","last_became_active":"2024-04-26T21:45:55.671562+0000","last_became_peered":"2024-04-26T21:45:55.671562+0000","last_unstale":"2024-04-26T21:45:57.694496+0000","last_undegraded":"2024-04-26T21:45:57.694496+0000","last_fullsized":"2024-04-26T21:45:57.694496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:47.590826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697702+0000","last_change":"2024-04-26T21:45:56.696591+0000","last_active":"2024-04-26T21:45:57.697702+0000","last_peered":"2024-04-26T21:45:57.697702+0000","last_clean":"2024-04-26T21:45:57.697702+0000","last_became_active":"2024-04-26T21:45:56.695741+0000","last_became_peered":"2024-04-26T21:45:56.695741+0000","last_unstale":"2024-04-26T21:45:57.697702+0000","last_undegraded":"2024-04-26T21:45:57.697702+0000","last_fullsized":"2024-04-26T21:45:57.697702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782611+0000","last_change":"2024-04-26T21:45:55.661721+0000","last_active":"2024-04-26T21:45:57.782611+0000","last_peered":"2024-04-26T21:45:57.782611+0000","last_clean":"2024-04-26T21:45:57.782611+0000","last_became_active":"2024-04-26T21:45:55.661317+0000","last_became_peered":"2024-04-26T21:45:55.661317+0000","last_unstale":"2024-04-26T21:45:57.782611+0000","last_undegraded":"2024-04-26T21:45:57.782611+0000","last_fullsized":"2024-04-26T21:45:57.782611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.698417+0000","last_change":"2024-04-26T21:45:56.690727+0000","last_active":"2024-04-26T21:45:57.698417+0000","last_peered":"2024-04-26T21:45:57.698417+0000","last_clean":"2024-04-26T21:45:57.698417+0000","last_became_active":"2024-04-26T21:45:56.689340+0000","last_became_peered":"2024-04-26T21:45:56.689340+0000","last_unstale":"2024-04-26T21:45:57.698417+0000","last_undegraded":"2024-04-26T21:45:57.698417+0000","last_fullsized":"2024-04-26T21:45:57.698417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:16.382523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688598+0000","last_change":"2024-04-26T21:45:55.667840+0000","last_active":"2024-04-26T21:45:57.688598+0000","last_peered":"2024-04-26T21:45:57.688598+0000","last_clean":"2024-04-26T21:45:57.688598+0000","last_became_active":"2024-04-26T21:45:55.667095+0000","last_became_peered":"2024-04-26T21:45:55.667095+0000","last_unstale":"2024-04-26T21:45:57.688598+0000","last_undegraded":"2024-04-26T21:45:57.688598+0000","last_fullsized":"2024-04-26T21:45:57.688598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786997+0000","last_change":"2024-04-26T21:45:56.724926+0000","last_active":"2024-04-26T21:45:57.786997+0000","last_peered":"2024-04-26T21:45:57.786997+0000","last_clean":"2024-04-26T21:45:57.786997+0000","last_became_active":"2024-04-26T21:45:56.723585+0000","last_became_peered":"2024-04-26T21:45:56.723585+0000","last_unstale":"2024-04-26T21:45:57.786997+0000","last_undegraded":"2024-04-26T21:45:57.786997+0000","last_fullsized":"2024-04-26T21:45:57.786997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:20.684761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782488+0000","last_change":"2024-04-26T21:45:55.679971+0000","last_active":"2024-04-26T21:45:57.782488+0000","last_peered":"2024-04-26T21:45:57.782488+0000","last_clean":"2024-04-26T21:45:57.782488+0000","last_became_active":"2024-04-26T21:45:55.679650+0000","last_became_peered":"2024-04-26T21:45:55.679650+0000","last_unstale":"2024-04-26T21:45:57.782488+0000","last_undegraded":"2024-04-26T21:45:57.782488+0000","last_fullsized":"2024-04-26T21:45:57.782488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697335+0000","last_change":"2024-04-26T21:45:56.707369+0000","last_active":"2024-04-26T21:45:57.697335+0000","last_peered":"2024-04-26T21:45:57.697335+0000","last_clean":"2024-04-26T21:45:57.697335+0000","last_became_active":"2024-04-26T21:45:56.707148+0000","last_became_peered":"2024-04-26T21:45:56.707148+0000","last_unstale":"2024-04-26T21:45:57.697335+0000","last_undegraded":"2024-04-26T21:45:57.697335+0000","last_fullsized":"2024-04-26T21:45:57.697335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:45.008570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696696+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:45:57.696696+0000","last_peered":"2024-04-26T21:45:57.696696+0000","last_clean":"2024-04-26T21:45:57.696696+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:45:57.696696+0000","last_undegraded":"2024-04-26T21:45:57.696696+0000","last_fullsized":"2024-04-26T21:45:57.696696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.785566+0000","last_change":"2024-04-26T21:45:55.671235+0000","last_active":"2024-04-26T21:45:57.785566+0000","last_peered":"2024-04-26T21:45:57.785566+0000","last_clean":"2024-04-26T21:45:57.785566+0000","last_became_active":"2024-04-26T21:45:55.670854+0000","last_became_peered":"2024-04-26T21:45:55.670854+0000","last_unstale":"2024-04-26T21:45:57.785566+0000","last_undegraded":"2024-04-26T21:45:57.785566+0000","last_fullsized":"2024-04-26T21:45:57.785566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783861+0000","last_change":"2024-04-26T21:45:56.723841+0000","last_active":"2024-04-26T21:45:57.783861+0000","last_peered":"2024-04-26T21:45:57.783861+0000","last_clean":"2024-04-26T21:45:57.783861+0000","last_became_active":"2024-04-26T21:45:56.723301+0000","last_became_peered":"2024-04-26T21:45:56.723301+0000","last_unstale":"2024-04-26T21:45:57.783861+0000","last_undegraded":"2024-04-26T21:45:57.783861+0000","last_fullsized":"2024-04-26T21:45:57.783861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:16.014367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696426+0000","last_change":"2024-04-26T21:45:55.669460+0000","last_active":"2024-04-26T21:45:57.696426+0000","last_peered":"2024-04-26T21:45:57.696426+0000","last_clean":"2024-04-26T21:45:57.696426+0000","last_became_active":"2024-04-26T21:45:55.668390+0000","last_became_peered":"2024-04-26T21:45:55.668390+0000","last_unstale":"2024-04-26T21:45:57.696426+0000","last_undegraded":"2024-04-26T21:45:57.696426+0000","last_fullsized":"2024-04-26T21:45:57.696426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783571+0000","last_change":"2024-04-26T21:45:56.700088+0000","last_active":"2024-04-26T21:45:57.783571+0000","last_peered":"2024-04-26T21:45:57.783571+0000","last_clean":"2024-04-26T21:45:57.783571+0000","last_became_active":"2024-04-26T21:45:56.699757+0000","last_became_peered":"2024-04-26T21:45:56.699757+0000","last_unstale":"2024-04-26T21:45:57.783571+0000","last_undegraded":"2024-04-26T21:45:57.783571+0000","last_fullsized":"2024-04-26T21:45:57.783571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:57.821808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.782410+0000","last_change":"2024-04-26T21:45:55.662154+0000","last_active":"2024-04-26T21:45:57.782410+0000","last_peered":"2024-04-26T21:45:57.782410+0000","last_clean":"2024-04-26T21:45:57.782410+0000","last_became_active":"2024-04-26T21:45:55.661829+0000","last_became_peered":"2024-04-26T21:45:55.661829+0000","last_unstale":"2024-04-26T21:45:57.782410+0000","last_undegraded":"2024-04-26T21:45:57.782410+0000","last_fullsized":"2024-04-26T21:45:57.782410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688590+0000","last_change":"2024-04-26T21:45:56.689249+0000","last_active":"2024-04-26T21:45:57.688590+0000","last_peered":"2024-04-26T21:45:57.688590+0000","last_clean":"2024-04-26T21:45:57.688590+0000","last_became_active":"2024-04-26T21:45:56.688918+0000","last_became_peered":"2024-04-26T21:45:56.688918+0000","last_unstale":"2024-04-26T21:45:57.688590+0000","last_undegraded":"2024-04-26T21:45:57.688590+0000","last_fullsized":"2024-04-26T21:45:57.688590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:30.947250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786809+0000","last_change":"2024-04-26T21:45:55.672652+0000","last_active":"2024-04-26T21:45:57.786809+0000","last_peered":"2024-04-26T21:45:57.786809+0000","last_clean":"2024-04-26T21:45:57.786809+0000","last_became_active":"2024-04-26T21:45:55.672091+0000","last_became_peered":"2024-04-26T21:45:55.672091+0000","last_unstale":"2024-04-26T21:45:57.786809+0000","last_undegraded":"2024-04-26T21:45:57.786809+0000","last_fullsized":"2024-04-26T21:45:57.786809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:12.199574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693856+0000","last_change":"2024-04-26T21:45:56.697429+0000","last_active":"2024-04-26T21:45:57.693856+0000","last_peered":"2024-04-26T21:45:57.693856+0000","last_clean":"2024-04-26T21:45:57.693856+0000","last_became_active":"2024-04-26T21:45:56.697080+0000","last_became_peered":"2024-04-26T21:45:56.697080+0000","last_unstale":"2024-04-26T21:45:57.693856+0000","last_undegraded":"2024-04-26T21:45:57.693856+0000","last_fullsized":"2024-04-26T21:45:57.693856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:50.881602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687872+0000","last_change":"2024-04-26T21:45:55.667311+0000","last_active":"2024-04-26T21:45:57.687872+0000","last_peered":"2024-04-26T21:45:57.687872+0000","last_clean":"2024-04-26T21:45:57.687872+0000","last_became_active":"2024-04-26T21:45:55.667025+0000","last_became_peered":"2024-04-26T21:45:55.667025+0000","last_unstale":"2024-04-26T21:45:57.687872+0000","last_undegraded":"2024-04-26T21:45:57.687872+0000","last_fullsized":"2024-04-26T21:45:57.687872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:58.848375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786703+0000","last_change":"2024-04-26T21:45:56.714921+0000","last_active":"2024-04-26T21:45:57.786703+0000","last_peered":"2024-04-26T21:45:57.786703+0000","last_clean":"2024-04-26T21:45:57.786703+0000","last_became_active":"2024-04-26T21:45:56.714330+0000","last_became_peered":"2024-04-26T21:45:56.714330+0000","last_unstale":"2024-04-26T21:45:57.786703+0000","last_undegraded":"2024-04-26T21:45:57.786703+0000","last_fullsized":"2024-04-26T21:45:57.786703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:27.718466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697756+0000","last_change":"2024-04-26T21:45:55.673285+0000","last_active":"2024-04-26T21:45:57.697756+0000","last_peered":"2024-04-26T21:45:57.697756+0000","last_clean":"2024-04-26T21:45:57.697756+0000","last_became_active":"2024-04-26T21:45:55.672987+0000","last_became_peered":"2024-04-26T21:45:55.672987+0000","last_unstale":"2024-04-26T21:45:57.697756+0000","last_undegraded":"2024-04-26T21:45:57.697756+0000","last_fullsized":"2024-04-26T21:45:57.697756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:30.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687819+0000","last_change":"2024-04-26T21:45:56.696370+0000","last_active":"2024-04-26T21:45:57.687819+0000","last_peered":"2024-04-26T21:45:57.687819+0000","last_clean":"2024-04-26T21:45:57.687819+0000","last_became_active":"2024-04-26T21:45:56.696073+0000","last_became_peered":"2024-04-26T21:45:56.696073+0000","last_unstale":"2024-04-26T21:45:57.687819+0000","last_undegraded":"2024-04-26T21:45:57.687819+0000","last_fullsized":"2024-04-26T21:45:57.687819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688658+0000","last_change":"2024-04-26T21:45:55.683832+0000","last_active":"2024-04-26T21:45:57.688658+0000","last_peered":"2024-04-26T21:45:57.688658+0000","last_clean":"2024-04-26T21:45:57.688658+0000","last_became_active":"2024-04-26T21:45:55.683577+0000","last_became_peered":"2024-04-26T21:45:55.683577+0000","last_unstale":"2024-04-26T21:45:57.688658+0000","last_undegraded":"2024-04-26T21:45:57.688658+0000","last_fullsized":"2024-04-26T21:45:57.688658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:03.555729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784223+0000","last_change":"2024-04-26T21:45:56.695282+0000","last_active":"2024-04-26T21:45:57.784223+0000","last_peered":"2024-04-26T21:45:57.784223+0000","last_clean":"2024-04-26T21:45:57.784223+0000","last_became_active":"2024-04-26T21:45:56.694899+0000","last_became_peered":"2024-04-26T21:45:56.694899+0000","last_unstale":"2024-04-26T21:45:57.784223+0000","last_undegraded":"2024-04-26T21:45:57.784223+0000","last_fullsized":"2024-04-26T21:45:57.784223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:03.109014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.706431+0000","last_change":"2024-04-26T21:45:55.684115+0000","last_active":"2024-04-26T21:45:57.706431+0000","last_peered":"2024-04-26T21:45:57.706431+0000","last_clean":"2024-04-26T21:45:57.706431+0000","last_became_active":"2024-04-26T21:45:55.683742+0000","last_became_peered":"2024-04-26T21:45:55.683742+0000","last_unstale":"2024-04-26T21:45:57.706431+0000","last_undegraded":"2024-04-26T21:45:57.706431+0000","last_fullsized":"2024-04-26T21:45:57.706431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783867+0000","last_change":"2024-04-26T21:45:56.710633+0000","last_active":"2024-04-26T21:45:57.783867+0000","last_peered":"2024-04-26T21:45:57.783867+0000","last_clean":"2024-04-26T21:45:57.783867+0000","last_became_active":"2024-04-26T21:45:56.710300+0000","last_became_peered":"2024-04-26T21:45:56.710300+0000","last_unstale":"2024-04-26T21:45:57.783867+0000","last_undegraded":"2024-04-26T21:45:57.783867+0000","last_fullsized":"2024-04-26T21:45:57.783867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:50.161980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.696388+0000","last_change":"2024-04-26T21:45:55.669182+0000","last_active":"2024-04-26T21:45:57.696388+0000","last_peered":"2024-04-26T21:45:57.696388+0000","last_clean":"2024-04-26T21:45:57.696388+0000","last_became_active":"2024-04-26T21:45:55.668065+0000","last_became_peered":"2024-04-26T21:45:55.668065+0000","last_unstale":"2024-04-26T21:45:57.696388+0000","last_undegraded":"2024-04-26T21:45:57.696388+0000","last_fullsized":"2024-04-26T21:45:57.696388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:49.876252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.693706+0000","last_change":"2024-04-26T21:45:56.690785+0000","last_active":"2024-04-26T21:45:57.693706+0000","last_peered":"2024-04-26T21:45:57.693706+0000","last_clean":"2024-04-26T21:45:57.693706+0000","last_became_active":"2024-04-26T21:45:56.690430+0000","last_became_peered":"2024-04-26T21:45:56.690430+0000","last_unstale":"2024-04-26T21:45:57.693706+0000","last_undegraded":"2024-04-26T21:45:57.693706+0000","last_fullsized":"2024-04-26T21:45:57.693706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:04.561079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.687976+0000","last_change":"2024-04-26T21:45:55.668369+0000","last_active":"2024-04-26T21:45:57.687976+0000","last_peered":"2024-04-26T21:45:57.687976+0000","last_clean":"2024-04-26T21:45:57.687976+0000","last_became_active":"2024-04-26T21:45:55.667944+0000","last_became_peered":"2024-04-26T21:45:55.667944+0000","last_unstale":"2024-04-26T21:45:57.687976+0000","last_undegraded":"2024-04-26T21:45:57.687976+0000","last_fullsized":"2024-04-26T21:45:57.687976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783372+0000","last_change":"2024-04-26T21:45:56.703067+0000","last_active":"2024-04-26T21:45:57.783372+0000","last_peered":"2024-04-26T21:45:57.783372+0000","last_clean":"2024-04-26T21:45:57.783372+0000","last_became_active":"2024-04-26T21:45:56.702744+0000","last_became_peered":"2024-04-26T21:45:56.702744+0000","last_unstale":"2024-04-26T21:45:57.783372+0000","last_undegraded":"2024-04-26T21:45:57.783372+0000","last_fullsized":"2024-04-26T21:45:57.783372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:04.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.688080+0000","last_change":"2024-04-26T21:45:55.669865+0000","last_active":"2024-04-26T21:45:57.688080+0000","last_peered":"2024-04-26T21:45:57.688080+0000","last_clean":"2024-04-26T21:45:57.688080+0000","last_became_active":"2024-04-26T21:45:55.669595+0000","last_became_peered":"2024-04-26T21:45:55.669595+0000","last_unstale":"2024-04-26T21:45:57.688080+0000","last_undegraded":"2024-04-26T21:45:57.688080+0000","last_fullsized":"2024-04-26T21:45:57.688080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:04.917559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783922+0000","last_change":"2024-04-26T21:45:56.699968+0000","last_active":"2024-04-26T21:45:57.783922+0000","last_peered":"2024-04-26T21:45:57.783922+0000","last_clean":"2024-04-26T21:45:57.783922+0000","last_became_active":"2024-04-26T21:45:56.699639+0000","last_became_peered":"2024-04-26T21:45:56.699639+0000","last_unstale":"2024-04-26T21:45:57.783922+0000","last_undegraded":"2024-04-26T21:45:57.783922+0000","last_fullsized":"2024-04-26T21:45:57.783922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:29.409931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697360+0000","last_change":"2024-04-26T21:45:55.673651+0000","last_active":"2024-04-26T21:45:57.697360+0000","last_peered":"2024-04-26T21:45:57.697360+0000","last_clean":"2024-04-26T21:45:57.697360+0000","last_became_active":"2024-04-26T21:45:55.673284+0000","last_became_peered":"2024-04-26T21:45:55.673284+0000","last_unstale":"2024-04-26T21:45:57.697360+0000","last_undegraded":"2024-04-26T21:45:57.697360+0000","last_fullsized":"2024-04-26T21:45:57.697360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:57.099549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.705814+0000","last_change":"2024-04-26T21:45:56.694260+0000","last_active":"2024-04-26T21:45:57.705814+0000","last_peered":"2024-04-26T21:45:57.705814+0000","last_clean":"2024-04-26T21:45:57.705814+0000","last_became_active":"2024-04-26T21:45:56.693917+0000","last_became_peered":"2024-04-26T21:45:56.693917+0000","last_unstale":"2024-04-26T21:45:57.705814+0000","last_undegraded":"2024-04-26T21:45:57.705814+0000","last_fullsized":"2024-04-26T21:45:57.705814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:53.852686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786302+0000","last_change":"2024-04-26T21:45:56.715202+0000","last_active":"2024-04-26T21:45:57.786302+0000","last_peered":"2024-04-26T21:45:57.786302+0000","last_clean":"2024-04-26T21:45:57.786302+0000","last_became_active":"2024-04-26T21:45:56.714700+0000","last_became_peered":"2024-04-26T21:45:56.714700+0000","last_unstale":"2024-04-26T21:45:57.786302+0000","last_undegraded":"2024-04-26T21:45:57.786302+0000","last_fullsized":"2024-04-26T21:45:57.786302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:30.102278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697171+0000","last_change":"2024-04-26T21:45:55.681370+0000","last_active":"2024-04-26T21:45:57.697171+0000","last_peered":"2024-04-26T21:45:57.697171+0000","last_clean":"2024-04-26T21:45:57.697171+0000","last_became_active":"2024-04-26T21:45:55.681034+0000","last_became_peered":"2024-04-26T21:45:55.681034+0000","last_unstale":"2024-04-26T21:45:57.697171+0000","last_undegraded":"2024-04-26T21:45:57.697171+0000","last_fullsized":"2024-04-26T21:45:57.697171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:05.692284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.786801+0000","last_change":"2024-04-26T21:45:56.713692+0000","last_active":"2024-04-26T21:45:57.786801+0000","last_peered":"2024-04-26T21:45:57.786801+0000","last_clean":"2024-04-26T21:45:57.786801+0000","last_became_active":"2024-04-26T21:45:56.713374+0000","last_became_peered":"2024-04-26T21:45:56.713374+0000","last_unstale":"2024-04-26T21:45:57.786801+0000","last_undegraded":"2024-04-26T21:45:57.786801+0000","last_fullsized":"2024-04-26T21:45:57.786801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:34.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.697482+0000","last_change":"2024-04-26T21:45:55.679436+0000","last_active":"2024-04-26T21:45:57.697482+0000","last_peered":"2024-04-26T21:45:57.697482+0000","last_clean":"2024-04-26T21:45:57.697482+0000","last_became_active":"2024-04-26T21:45:55.678851+0000","last_became_peered":"2024-04-26T21:45:55.678851+0000","last_unstale":"2024-04-26T21:45:57.697482+0000","last_undegraded":"2024-04-26T21:45:57.697482+0000","last_fullsized":"2024-04-26T21:45:57.697482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:16.328469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783868+0000","last_change":"2024-04-26T21:45:56.719859+0000","last_active":"2024-04-26T21:45:57.783868+0000","last_peered":"2024-04-26T21:45:57.783868+0000","last_clean":"2024-04-26T21:45:57.783868+0000","last_became_active":"2024-04-26T21:45:56.719166+0000","last_became_peered":"2024-04-26T21:45:56.719166+0000","last_unstale":"2024-04-26T21:45:57.783868+0000","last_undegraded":"2024-04-26T21:45:57.783868+0000","last_fullsized":"2024-04-26T21:45:57.783868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:55.964359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784767+0000","last_change":"2024-04-26T21:45:55.664913+0000","last_active":"2024-04-26T21:45:57.784767+0000","last_peered":"2024-04-26T21:45:57.784767+0000","last_clean":"2024-04-26T21:45:57.784767+0000","last_became_active":"2024-04-26T21:45:55.664561+0000","last_became_peered":"2024-04-26T21:45:55.664561+0000","last_unstale":"2024-04-26T21:45:57.784767+0000","last_undegraded":"2024-04-26T21:45:57.784767+0000","last_fullsized":"2024-04-26T21:45:57.784767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.784594+0000","last_change":"2024-04-26T21:45:56.698358+0000","last_active":"2024-04-26T21:45:57.784594+0000","last_peered":"2024-04-26T21:45:57.784594+0000","last_clean":"2024-04-26T21:45:57.784594+0000","last_became_active":"2024-04-26T21:45:56.697924+0000","last_became_peered":"2024-04-26T21:45:56.697924+0000","last_unstale":"2024-04-26T21:45:57.784594+0000","last_undegraded":"2024-04-26T21:45:57.784594+0000","last_fullsized":"2024-04-26T21:45:57.784594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:55.629861+0000","last_clean_scrub_stamp":"2024-04-26T21:45:55.629861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:43.112971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:45:57.783992+0000","last_change":"2024-04-26T21:45:55.668737+0000","last_active":"2024-04-26T21:45:57.783992+0000","last_peered":"2024-04-26T21:45:57.783992+0000","last_clean":"2024-04-26T21:45:57.783992+0000","last_became_active":"2024-04-26T21:45:55.668415+0000","last_became_peered":"2024-04-26T21:45:55.668415+0000","last_unstale":"2024-04-26T21:45:57.783992+0000","last_undegraded":"2024-04-26T21:45:57.783992+0000","last_fullsized":"2024-04-26T21:45:57.783992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:54.624511+0000","last_clean_scrub_stamp":"2024-04-26T21:45:54.624511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:50.322253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:46:07.050 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:46:07.050 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:46:07.050 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:46:07.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:46:07.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.284+0000 7f82db400640 1 Processor -- start 2024-04-26T21:46:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.284+0000 7f82db400640 1 -- start start 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82db400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 0x7f82d409be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82db400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82d40b58a0 0x7f82d406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82db400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d409c370 con 0x7f82d406f230 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82db400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f82d4070420 con 0x7f82d40b58a0 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82db400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f82d406e790 con 0x7f82d409ba50 2024-04-26T21:46:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82d3e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82d40b58a0 0x7f82d406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82d9400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82d406f230 0x7f82d406d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58978/0 (socket says 172.21.15.106:58978) 2024-04-26T21:46:07.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.286+0000 7f82d9400640 1 -- 172.21.15.106:0/3391095916 learned_addr learned my addr 172.21.15.106:0/3391095916 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d8a00640 1 --2- 172.21.15.106:0/3391095916 >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 0x7f82d409be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d3e00640 1 -- 172.21.15.106:0/3391095916 >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 msgr2=0x7f82d409be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d3e00640 1 --2- 172.21.15.106:0/3391095916 >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 0x7f82d409be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d3e00640 1 -- 172.21.15.106:0/3391095916 >> v1:172.21.15.106:6789/0 conn(0x7f82d406f230 legacy=0x7f82d406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d3e00640 1 -- 172.21.15.106:0/3391095916 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d4070670 con 0x7f82d40b58a0 2024-04-26T21:46:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d3400640 1 -- 172.21.15.106:0/3391095916 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940093775 0 0) 0x7f82d409c370 con 0x7f82d406f230 2024-04-26T21:46:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.287+0000 7f82d8a00640 1 --2- 172.21.15.106:0/3391095916 >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 0x7f82d409be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.288+0000 7f82d3e00640 1 --2- 172.21.15.106:0/3391095916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d40b58a0 0x7f82d406d460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82d40767a0 tx=0x7f82c402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b1282298bff9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.288+0000 7f82d3400640 1 -- 172.21.15.106:0/3391095916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4007740 con 0x7f82d40b58a0 2024-04-26T21:46:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.288+0000 7f82d3400640 1 -- 172.21.15.106:0/3391095916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c40078e0 con 0x7f82d40b58a0 2024-04-26T21:46:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.288+0000 7f82d3400640 1 -- 172.21.15.106:0/3391095916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4007cb0 con 0x7f82d40b58a0 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 -- 172.21.15.106:0/3391095916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d40b58a0 msgr2=0x7f82d406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 --2- 172.21.15.106:0/3391095916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d40b58a0 0x7f82d406d460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82d40767a0 tx=0x7f82c402a5f0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 -- 172.21.15.106:0/3391095916 shutdown_connections 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 --2- 172.21.15.106:0/3391095916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d40b58a0 0x7f82d406d460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 --2- 172.21.15.106:0/3391095916 >> v2:172.21.15.204:3300/0 conn(0x7f82d409ba50 0x7f82d409be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 -- 172.21.15.106:0/3391095916 >> 172.21.15.106:0/3391095916 conn(0x7f82d40b4b70 msgr2=0x7f82d40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 -- 172.21.15.106:0/3391095916 shutdown_connections 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.289+0000 7f82db400640 1 -- 172.21.15.106:0/3391095916 wait complete. 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.291+0000 7f82db400640 1 Processor -- start 2024-04-26T21:46:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.291+0000 7f82db400640 1 -- start start 2024-04-26T21:46:07.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82d409ba50 0x7f82d41d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d40b58a0 0x7f82d41d8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82d406ecc0 con 0x7f82d406f230 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82d406e9e0 con 0x7f82d40b58a0 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82db400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40cf440 con 0x7f82d409ba50 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52376/0 (socket says 172.21.15.106:52376) 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 -- 172.21.15.106:0/510325423 learned_addr learned my addr 172.21.15.106:0/510325423 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d3e00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82d409ba50 0x7f82d41d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d9400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d40b58a0 0x7f82d41d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 -- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82d409ba50 msgr2=0x7f82d41d7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82d409ba50 0x7f82d41d7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 -- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d40b58a0 msgr2=0x7f82d41d8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d40b58a0 0x7f82d41d8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.292+0000 7f82d8a00640 1 -- 172.21.15.106:0/510325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40cc330 con 0x7f82d406f230 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82d8a00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c7f0 tx=0x7f82bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc015070 con 0x7f82d406f230 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc007820 con 0x7f82d406f230 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc007bd0 con 0x7f82d406f230 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d40cc610 con 0x7f82d406f230 2024-04-26T21:46:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.293+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d40ccbd0 con 0x7f82d406f230 2024-04-26T21:46:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.294+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d4069540 con 0x7f82d406f230 2024-04-26T21:46:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.295+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82bc018070 con 0x7f82d406f230 2024-04-26T21:46:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.296+0000 7f82d0c00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 0x7f82b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.296+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f82bc08bc70 con 0x7f82d406f230 2024-04-26T21:46:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.296+0000 7f82d3e00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 0x7f82b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.297+0000 7f82d3e00640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 0x7f82b006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82c4034000 tx=0x7f82c402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.300+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82bc05a090 con 0x7f82d406f230 2024-04-26T21:46:07.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.523+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82d40ccfb0 con 0x7f82d406f230 2024-04-26T21:46:07.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.524+0000 7f82d0c00640 1 -- 172.21.15.106:0/510325423 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f82bc059990 con 0x7f82d406f230 2024-04-26T21:46:07.524 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:07.524 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 msgr2=0x7f82b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 0x7f82b006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82c4034000 tx=0x7f82c402f000 comp rx=0 tx=0).stop 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 msgr2=0x7f82d41d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c7f0 tx=0x7f82bc00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 shutdown_connections 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d40b58a0 0x7f82d41d8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82b006c8f0 0x7f82b006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82d409ba50 0x7f82d41d7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 --2- 172.21.15.106:0/510325423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82d406f230 0x7f82d41d5850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 >> 172.21.15.106:0/510325423 conn(0x7f82d40b4b70 msgr2=0x7f82d40d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 shutdown_connections 2024-04-26T21:46:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.527+0000 7f82db400640 1 -- 172.21.15.106:0/510325423 wait complete. 2024-04-26T21:46:07.544 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:46:07.544 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.745+0000 7fb3bec00640 1 Processor -- start 2024-04-26T21:46:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.745+0000 7fb3bec00640 1 -- start start 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.746+0000 7fb3bec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 0x7fb3b80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3bec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb3b80cd650 0x7fb3b80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3bec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b80af9a0 con 0x7fb3b815cdf0 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3bec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b80ae2c0 con 0x7fb3b80cd650 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3bec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b80b05b0 con 0x7fb3b806ce80 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3af400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 0x7fb3b80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3af400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 0x7fb3b80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:39802/0 (socket says 172.21.15.106:39802) 2024-04-26T21:46:07.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3af400640 1 -- 172.21.15.106:0/138993828 learned_addr learned my addr 172.21.15.106:0/138993828 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:07.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.747+0000 7fb3bcc00640 1 --2- 172.21.15.106:0/138993828 >> v2:172.21.15.204:3300/0 conn(0x7fb3b80cd650 0x7fb3b80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.748+0000 7fb3bcc00640 1 -- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 msgr2=0x7fb3b80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.748+0000 7fb3bcc00640 1 --2- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 0x7fb3b80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.748+0000 7fb3bcc00640 1 -- 172.21.15.106:0/138993828 >> v1:172.21.15.204:6789/0 conn(0x7fb3b815cdf0 legacy=0x7fb3b806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.748+0000 7fb3bcc00640 1 -- 172.21.15.106:0/138993828 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80afc80 con 0x7fb3b80cd650 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.750+0000 7fb3bcc00640 1 --2- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b80b3d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d90 tx=0x7fb3b002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7ccaec75ed2398 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.750+0000 7fb3b7400640 1 -- 172.21.15.106:0/138993828 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007740 con 0x7fb3b80cd650 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.750+0000 7fb3b7400640 1 -- 172.21.15.106:0/138993828 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b00078e0 con 0x7fb3b80cd650 2024-04-26T21:46:07.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.750+0000 7fb3b7400640 1 -- 172.21.15.106:0/138993828 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007cb0 con 0x7fb3b80cd650 2024-04-26T21:46:07.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 -- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 msgr2=0x7fb3b80b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 --2- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b80b3d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d90 tx=0x7fb3b002a660 comp rx=0 tx=0).stop 2024-04-26T21:46:07.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 -- 172.21.15.106:0/138993828 shutdown_connections 2024-04-26T21:46:07.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 --2- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b80b3d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 --2- 172.21.15.106:0/138993828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b806ce80 0x7fb3b80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 -- 172.21.15.106:0/138993828 >> 172.21.15.106:0/138993828 conn(0x7fb3b80b2d50 msgr2=0x7fb3b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 -- 172.21.15.106:0/138993828 shutdown_connections 2024-04-26T21:46:07.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.751+0000 7fb3bec00640 1 -- 172.21.15.106:0/138993828 wait complete. 2024-04-26T21:46:07.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.754+0000 7fb3bec00640 1 Processor -- start 2024-04-26T21:46:07.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.754+0000 7fb3bec00640 1 -- start start 2024-04-26T21:46:07.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b814d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 0x7fb3b809c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b815d1d0 con 0x7fb3b806ce80 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80b0240 con 0x7fb3b80cd650 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3bec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b815d350 con 0x7fb3b815cdf0 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52380/0 (socket says 172.21.15.106:52380) 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.755+0000 7fb3b7e00640 1 -- 172.21.15.106:0/489916142 learned_addr learned my addr 172.21.15.106:0/489916142 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3bcc00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 0x7fb3b809c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3af400640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b814d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 -- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 msgr2=0x7fb3b809c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 0x7fb3b809c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 -- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 msgr2=0x7fb3b814d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b814d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 -- 172.21.15.106:0/489916142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b809e0d0 con 0x7fb3b806ce80 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3bcc00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 0x7fb3b809c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3af400640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b814d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b7e00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c430 tx=0x7fb3a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8017070 con 0x7fb3b806ce80 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a80052d0 con 0x7fb3b806ce80 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003830 con 0x7fb3b806ce80 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.756+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b809e3b0 con 0x7fb3b806ce80 2024-04-26T21:46:07.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.757+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b8151780 con 0x7fb3b806ce80 2024-04-26T21:46:07.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.758+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b8151980 con 0x7fb3b806ce80 2024-04-26T21:46:07.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.759+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3a801a070 con 0x7fb3b806ce80 2024-04-26T21:46:07.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.760+0000 7fb3b4c00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 0x7fb39406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.760+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb3a808ae90 con 0x7fb3b806ce80 2024-04-26T21:46:07.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.760+0000 7fb3af400640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 0x7fb39406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.761+0000 7fb3af400640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 0x7fb39406ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80cd390 tx=0x7fb3a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.764+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3b8151980 con 0x7fb3b806ce80 2024-04-26T21:46:07.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.965+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3b80b1810 con 0x7fb3b806ce80 2024-04-26T21:46:07.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.968+0000 7fb3b4c00640 1 -- 172.21.15.106:0/489916142 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fb3b80b1810 con 0x7fb3b806ce80 2024-04-26T21:46:07.969 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:07.969 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 msgr2=0x7fb39406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 0x7fb39406ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80cd390 tx=0x7fb3a0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:07.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 msgr2=0x7fb3b809daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c430 tx=0x7fb3a800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 shutdown_connections 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb39406c920 0x7fb39406ede0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b815cdf0 0x7fb3b809c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b80cd650 0x7fb3b814d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 --2- 172.21.15.106:0/489916142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3b806ce80 0x7fb3b809daf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 >> 172.21.15.106:0/489916142 conn(0x7fb3b80b2d50 msgr2=0x7fb3b806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 shutdown_connections 2024-04-26T21:46:07.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:07.971+0000 7fb3bec00640 1 -- 172.21.15.106:0/489916142 wait complete. 2024-04-26T21:46:07.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:07.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:08.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.181+0000 7fb48e400640 1 Processor -- start 2024-04-26T21:46:08.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.182+0000 7fb48e400640 1 -- start start 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.183+0000 7fb48e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb4880d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.183+0000 7fb48e400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4880d1da0 0x7fb488162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.183+0000 7fb48e400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4880d3830 con 0x7fb4880b6170 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.183+0000 7fb48e400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb4880d40d0 con 0x7fb4880d1da0 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.183+0000 7fb48e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb4880d3b10 con 0x7fb4880d19c0 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb47ea00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb4880b6170 0x7fb4880c8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36366/0 (socket says 172.21.15.106:36366) 2024-04-26T21:46:08.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb47ea00640 1 -- 172.21.15.106:0/564960957 learned_addr learned my addr 172.21.15.106:0/564960957 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:08.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb487400640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb4880d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb487e00640 1 --2- 172.21.15.106:0/564960957 >> v2:172.21.15.204:3300/0 conn(0x7fb4880d1da0 0x7fb488162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269862972 0 0) 0x7fb4880d3830 con 0x7fb4880b6170 2024-04-26T21:46:08.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb46c003520 con 0x7fb4880b6170 2024-04-26T21:46:08.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 866754660 0 0) 0x7fb46c003520 con 0x7fb4880b6170 2024-04-26T21:46:08.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 msgr2=0x7fb4880d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb4880d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.184+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 >> v2:172.21.15.204:3300/0 conn(0x7fb4880d1da0 msgr2=0x7fb488162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 --2- 172.21.15.106:0/564960957 >> v2:172.21.15.204:3300/0 conn(0x7fb4880d1da0 0x7fb488162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4880d3df0 con 0x7fb4880b6170 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fb470001670 con 0x7fb4880b6170 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 >> v1:172.21.15.106:6789/0 conn(0x7fb4880b6170 legacy=0x7fb4880c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb46c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c003f80 0x7fb46c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb46c004d40 0x7fb46c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb4880d3b10 con 0x7fb46c004d40 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00b9f0 con 0x7fb46c003f80 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c00bbf0 con 0x7fb4880d19c0 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb46c004d40 0x7fb46c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb47ea00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c003f80 0x7fb46c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487400640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb46c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 -- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 msgr2=0x7fb46c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880d19c0 0x7fb46c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 -- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c003f80 msgr2=0x7fb46c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c003f80 0x7fb46c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb487e00640 1 -- 172.21.15.106:0/564960957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c003520 con 0x7fb46c004d40 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.185+0000 7fb47ea00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c003f80 0x7fb46c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.186+0000 7fb487e00640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb46c004d40 0x7fb46c0051c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb480009fc0 tx=0x7fb46c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e453401319ddfad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480009c10 con 0x7fb46c004d40 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb480036070 con 0x7fb46c004d40 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb486a00640 1 -- 172.21.15.106:0/564960957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48002d070 con 0x7fb46c004d40 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 -- 172.21.15.106:0/564960957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb46c004d40 msgr2=0x7fb46c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 --2- 172.21.15.106:0/564960957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb46c004d40 0x7fb46c0051c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb480009fc0 tx=0x7fb46c005830 comp rx=0 tx=0).stop 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb487e00640 1 -- 172.21.15.106:0/564960957 reap_dead start 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 -- 172.21.15.106:0/564960957 shutdown_connections 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 -- 172.21.15.106:0/564960957 >> 172.21.15.106:0/564960957 conn(0x7fb48809e460 msgr2=0x7fb4880d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 -- 172.21.15.106:0/564960957 shutdown_connections 2024-04-26T21:46:08.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.187+0000 7fb48e400640 1 -- 172.21.15.106:0/564960957 wait complete. 2024-04-26T21:46:08.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.189+0000 7fb48e400640 1 Processor -- start 2024-04-26T21:46:08.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.190+0000 7fb48e400640 1 -- start start 2024-04-26T21:46:08.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c004d40 0x7fb48809f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 0x7fb4880b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb488162d00 con 0x7fb4881f7f60 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb488162b80 con 0x7fb46c004d40 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb48e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb488162e80 con 0x7fb4880b7d40 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34992/0 (socket says 172.21.15.106:34992) 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 -- 172.21.15.106:0/3668597036 learned_addr learned my addr 172.21.15.106:0/3668597036 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb487e00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 0x7fb4880b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb487400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c004d40 0x7fb48809f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 -- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 msgr2=0x7fb4880b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 0x7fb4880b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 -- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c004d40 msgr2=0x7fb48809f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c004d40 0x7fb48809f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb47ea00640 1 -- 172.21.15.106:0/3668597036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4880b8a00 con 0x7fb4881f7f60 2024-04-26T21:46:08.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.191+0000 7fb487e00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 0x7fb4880b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:08.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb47ea00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb47000d910 tx=0x7fb47000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470019070 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4700074e0 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700049e0 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4880b8ce0 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.192+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4880b92a0 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.193+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb488069650 con 0x7fb4881f7f60 2024-04-26T21:46:08.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.194+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb470004b80 con 0x7fb4881f7f60 2024-04-26T21:46:08.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.195+0000 7fb47fe00640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 0x7fb46406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.195+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb47008b6c0 con 0x7fb4881f7f60 2024-04-26T21:46:08.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.195+0000 7fb487400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 0x7fb46406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.196+0000 7fb487400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 0x7fb46406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb478009a10 tx=0x7fb478008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.200+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb470059a50 con 0x7fb4881f7f60 2024-04-26T21:46:08.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.403+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4880d26d0 con 0x7fb4881f7f60 2024-04-26T21:46:08.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.403+0000 7fb47fe00640 1 -- 172.21.15.106:0/3668597036 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb470059350 con 0x7fb4881f7f60 2024-04-26T21:46:08.404 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 msgr2=0x7fb46406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 0x7fb46406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb478009a10 tx=0x7fb478008040 comp rx=0 tx=0).stop 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 msgr2=0x7fb4881f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb47000d910 tx=0x7fb47000ddd0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 shutdown_connections 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb46406c8f0 0x7fb46406edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4880b7d40 0x7fb4880b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4881f7f60 0x7fb4881f83e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 --2- 172.21.15.106:0/3668597036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c004d40 0x7fb48809f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 >> 172.21.15.106:0/3668597036 conn(0x7fb48809e460 msgr2=0x7fb4880d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 shutdown_connections 2024-04-26T21:46:08.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.406+0000 7fb48e400640 1 -- 172.21.15.106:0/3668597036 wait complete. 2024-04-26T21:46:08.425 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.629+0000 7f0113e00640 1 Processor -- start 2024-04-26T21:46:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.629+0000 7f0113e00640 1 -- start start 2024-04-26T21:46:08.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0113e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01140cfb90 0x7f01140d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0113e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140d2c50 0x7f01140d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0113e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01140d0170 con 0x7f011415f7f0 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0113e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f01140d0450 con 0x7f01140cfb90 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0113e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01140d0730 con 0x7f01140d2c50 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f010be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01140cfb90 0x7f01140d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0112a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f011415f7f0 0x7f011415fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36378/0 (socket says 172.21.15.106:36378) 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.631+0000 7f0112a00640 1 -- 172.21.15.106:0/2518235781 learned_addr learned my addr 172.21.15.106:0/2518235781 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f0113400640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140d2c50 0x7f01140d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f010be00640 1 -- 172.21.15.106:0/2518235781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140d2c50 msgr2=0x7f01140d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f010be00640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140d2c50 0x7f01140d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f010be00640 1 -- 172.21.15.106:0/2518235781 >> v1:172.21.15.106:6789/0 conn(0x7f011415f7f0 legacy=0x7f011415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f0112000640 1 -- 172.21.15.106:0/2518235781 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160043717 0 0) 0x7f01140d0170 con 0x7f011415f7f0 2024-04-26T21:46:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.632+0000 7f010be00640 1 -- 172.21.15.106:0/2518235781 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01140d0a10 con 0x7f01140cfb90 2024-04-26T21:46:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.633+0000 7f010be00640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01140cfb90 0x7f01140d22d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0097a0 tx=0x7f00fc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3abee814ac1a95d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.633+0000 7f0112000640 1 -- 172.21.15.106:0/2518235781 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc037070 con 0x7f01140cfb90 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.633+0000 7f0112000640 1 -- 172.21.15.106:0/2518235781 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc02ae50 con 0x7f01140cfb90 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.633+0000 7f0112000640 1 -- 172.21.15.106:0/2518235781 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc032660 con 0x7f01140cfb90 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 -- 172.21.15.106:0/2518235781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01140cfb90 msgr2=0x7f01140d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01140cfb90 0x7f01140d22d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0097a0 tx=0x7f00fc02a100 comp rx=0 tx=0).stop 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 -- 172.21.15.106:0/2518235781 shutdown_connections 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140d2c50 0x7f01140d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 --2- 172.21.15.106:0/2518235781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01140cfb90 0x7f01140d22d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 -- 172.21.15.106:0/2518235781 >> 172.21.15.106:0/2518235781 conn(0x7f01140d4f80 msgr2=0x7f01140d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 -- 172.21.15.106:0/2518235781 shutdown_connections 2024-04-26T21:46:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.634+0000 7f0113e00640 1 -- 172.21.15.106:0/2518235781 wait complete. 2024-04-26T21:46:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.636+0000 7f0113e00640 1 Processor -- start 2024-04-26T21:46:08.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.636+0000 7f0113e00640 1 -- start start 2024-04-26T21:46:08.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140cfb90 0x7f01140ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 0x7f01140cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f011415f430 con 0x7f011415f7f0 2024-04-26T21:46:08.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01140d2810 con 0x7f01140d2c50 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f011415f5b0 con 0x7f01140cfb90 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35018/0 (socket says 172.21.15.106:35018) 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113400640 1 -- 172.21.15.106:0/720280551 learned_addr learned my addr 172.21.15.106:0/720280551 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f010be00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 0x7f01140cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0112a00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140cfb90 0x7f01140ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113400640 1 -- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140cfb90 msgr2=0x7f01140ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.637+0000 7f0113400640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140cfb90 0x7f01140ba1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113400640 1 -- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 msgr2=0x7f01140cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113400640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 0x7f01140cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113400640 1 -- 172.21.15.106:0/720280551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0114154860 con 0x7f011415f7f0 2024-04-26T21:46:08.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f010be00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 0x7f01140cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:08.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113400640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f010c007e60 tx=0x7f010c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c014420 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c019070 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01140cd760 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c014820 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.638+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01140cd480 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.639+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01140693c0 con 0x7f011415f7f0 2024-04-26T21:46:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.640+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f010c0149c0 con 0x7f011415f7f0 2024-04-26T21:46:08.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.641+0000 7f010b400640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 0x7f00f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.641+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f010c08c3f0 con 0x7f011415f7f0 2024-04-26T21:46:08.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.642+0000 7f0112a00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 0x7f00f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.642+0000 7f0112a00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 0x7f00f006ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0104007fd0 tx=0x7f0104009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.646+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0114154860 con 0x7f011415f7f0 2024-04-26T21:46:08.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.845+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f011408ea90 con 0x7f011415f7f0 2024-04-26T21:46:08.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.846+0000 7f010b400640 1 -- 172.21.15.106:0/720280551 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f010c05a780 con 0x7f011415f7f0 2024-04-26T21:46:08.846 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:08.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 msgr2=0x7f00f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 0x7f00f006ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0104007fd0 tx=0x7f0104009040 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 msgr2=0x7f01140b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f010c007e60 tx=0x7f010c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 shutdown_connections 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f011415f7f0 0x7f01140b9250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01140d2c50 0x7f01140cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00f006c920 0x7f00f006ede0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 --2- 172.21.15.106:0/720280551 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01140cfb90 0x7f01140ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.848+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 >> 172.21.15.106:0/720280551 conn(0x7f01140d4f80 msgr2=0x7f0114160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.849+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 shutdown_connections 2024-04-26T21:46:08.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:08.849+0000 7f0113e00640 1 -- 172.21.15.106:0/720280551 wait complete. 2024-04-26T21:46:08.868 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.071+0000 7f14f7e00640 1 Processor -- start 2024-04-26T21:46:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.071+0000 7f14f7e00640 1 -- start start 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f14f80d1470 0x7f14f8167da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f7e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f8160b20 con 0x7f14f81467a0 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f8160e00 con 0x7f14f80b5ca0 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f7e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f14f8146b80 con 0x7f14f80d1470 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f6a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f14f81467a0 0x7f14f815dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36392/0 (socket says 172.21.15.106:36392) 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f14f80d1470 0x7f14f8167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.073+0000 7f14f6a00640 1 -- 172.21.15.106:0/384820600 learned_addr learned my addr 172.21.15.106:0/384820600 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:09.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.074+0000 7f14f7400640 1 -- 172.21.15.106:0/384820600 >> v1:172.21.15.204:6789/0 conn(0x7f14f80b5ca0 legacy=0x7f14f80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.074+0000 7f14f7400640 1 -- 172.21.15.106:0/384820600 >> v1:172.21.15.106:6789/0 conn(0x7f14f81467a0 legacy=0x7f14f815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.074+0000 7f14f7400640 1 -- 172.21.15.106:0/384820600 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f81610e0 con 0x7f14f80d1470 2024-04-26T21:46:09.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.074+0000 7f14f6000640 1 -- 172.21.15.106:0/384820600 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406765721 0 0) 0x7f14f8160b20 con 0x7f14f81467a0 2024-04-26T21:46:09.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.075+0000 7f14f7400640 1 --2- 172.21.15.106:0/384820600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d1470 0x7f14f8167da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14f8146d00 tx=0x7f14f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adf82e36498068b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.075+0000 7f14f6000640 1 -- 172.21.15.106:0/384820600 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0007740 con 0x7f14f80d1470 2024-04-26T21:46:09.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.075+0000 7f14f6000640 1 -- 172.21.15.106:0/384820600 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f00078e0 con 0x7f14f80d1470 2024-04-26T21:46:09.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.075+0000 7f14f6000640 1 -- 172.21.15.106:0/384820600 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0007cb0 con 0x7f14f80d1470 2024-04-26T21:46:09.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 -- 172.21.15.106:0/384820600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d1470 msgr2=0x7f14f8167da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 --2- 172.21.15.106:0/384820600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d1470 0x7f14f8167da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14f8146d00 tx=0x7f14f002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 -- 172.21.15.106:0/384820600 shutdown_connections 2024-04-26T21:46:09.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 --2- 172.21.15.106:0/384820600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80d1470 0x7f14f8167da0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 -- 172.21.15.106:0/384820600 >> 172.21.15.106:0/384820600 conn(0x7f14f806c530 msgr2=0x7f14f806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 -- 172.21.15.106:0/384820600 shutdown_connections 2024-04-26T21:46:09.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.076+0000 7f14f7e00640 1 -- 172.21.15.106:0/384820600 wait complete. 2024-04-26T21:46:09.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.078+0000 7f14f7e00640 1 Processor -- start 2024-04-26T21:46:09.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.078+0000 7f14f7e00640 1 -- start start 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 0x7f14f80abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f81467a0 0x7f14f80afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80b5100 con 0x7f14f80b5ca0 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8161730 con 0x7f14f81467a0 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80b5280 con 0x7f14f80d1470 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35044/0 (socket says 172.21.15.106:35044) 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 -- 172.21.15.106:0/1347367356 learned_addr learned my addr 172.21.15.106:0/1347367356 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14efe00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 0x7f14f80abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 -- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 msgr2=0x7f14f80abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f7400640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f81467a0 0x7f14f80afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 0x7f14f80abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 -- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f81467a0 msgr2=0x7f14f80afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f81467a0 0x7f14f80afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14f6a00640 1 -- 172.21.15.106:0/1347367356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f81f6400 con 0x7f14f80b5ca0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.079+0000 7f14efe00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 0x7f14f80abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14f6a00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c430 tx=0x7f14e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80074e0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e80047e0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004bd0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f81f66e0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.080+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81f6ca0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.081+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f80693c0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.082+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14e8011070 con 0x7f14f80b5ca0 2024-04-26T21:46:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.083+0000 7f14eea00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 0x7f14d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.083+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14e808b750 con 0x7f14f80b5ca0 2024-04-26T21:46:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.083+0000 7f14efe00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 0x7f14d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.084+0000 7f14efe00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 0x7f14d406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14f80cd940 tx=0x7f14e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.087+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14e8059b70 con 0x7f14f80b5ca0 2024-04-26T21:46:09.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.285+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14f806f7a0 con 0x7f14f80b5ca0 2024-04-26T21:46:09.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.286+0000 7f14eea00640 1 -- 172.21.15.106:0/1347367356 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f14e8059470 con 0x7f14f80b5ca0 2024-04-26T21:46:09.286 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 msgr2=0x7f14d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 0x7f14d406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14f80cd940 tx=0x7f14e0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 msgr2=0x7f14f815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c430 tx=0x7f14e800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 shutdown_connections 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f81467a0 0x7f14f80afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f14d406c8f0 0x7f14d406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14f80d1470 0x7f14f80abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 --2- 172.21.15.106:0/1347367356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14f80b5ca0 0x7f14f815c4a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 >> 172.21.15.106:0/1347367356 conn(0x7f14f806c530 msgr2=0x7f14f80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 shutdown_connections 2024-04-26T21:46:09.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.289+0000 7f14f7e00640 1 -- 172.21.15.106:0/1347367356 wait complete. 2024-04-26T21:46:09.307 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:09.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.490+0000 7ff477200640 1 Processor -- start 2024-04-26T21:46:09.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.490+0000 7ff477200640 1 -- start start 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff477200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff4700d0e70 0x7ff4700d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff477200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff4700b3b20 0x7ff4700b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff477200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff470146a60 con 0x7ff470146680 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff477200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff47006e970 con 0x7ff4700b3b20 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff477200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff4700b60a0 con 0x7ff4700d0e70 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff4700b3b20 0x7ff4700b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff4700b3b20 0x7ff4700b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35054/0 (socket says 172.21.15.106:35054) 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 -- 172.21.15.106:0/1123656976 learned_addr learned my addr 172.21.15.106:0/1123656976 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff475400640 1 --2- 172.21.15.106:0/1123656976 >> v2:172.21.15.204:3300/0 conn(0x7ff4700d0e70 0x7ff4700d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 -- 172.21.15.106:0/1123656976 >> v2:172.21.15.204:3300/0 conn(0x7ff4700d0e70 msgr2=0x7ff4700d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 --2- 172.21.15.106:0/1123656976 >> v2:172.21.15.204:3300/0 conn(0x7ff4700d0e70 0x7ff4700d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 -- 172.21.15.106:0/1123656976 >> v1:172.21.15.204:6789/0 conn(0x7ff470146680 legacy=0x7ff4700b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.492+0000 7ff476800640 1 -- 172.21.15.106:0/1123656976 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47006dd60 con 0x7ff4700b3b20 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.493+0000 7ff476800640 1 --2- 172.21.15.106:0/1123656976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4700b3b20 0x7ff4700b3f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff470160890 tx=0x7ff46c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94f8e6ac6e3bdcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.494+0000 7ff474a00640 1 -- 172.21.15.106:0/1123656976 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c037070 con 0x7ff4700b3b20 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.494+0000 7ff474a00640 1 -- 172.21.15.106:0/1123656976 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c032440 con 0x7ff4700b3b20 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.494+0000 7ff474a00640 1 -- 172.21.15.106:0/1123656976 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c032810 con 0x7ff4700b3b20 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.494+0000 7ff477200640 1 -- 172.21.15.106:0/1123656976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4700b3b20 msgr2=0x7ff4700b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.494+0000 7ff477200640 1 --2- 172.21.15.106:0/1123656976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4700b3b20 0x7ff4700b3f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff470160890 tx=0x7ff46c02a230 comp rx=0 tx=0).stop 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 -- 172.21.15.106:0/1123656976 shutdown_connections 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 --2- 172.21.15.106:0/1123656976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4700b3b20 0x7ff4700b3f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 --2- 172.21.15.106:0/1123656976 >> v2:172.21.15.204:3300/0 conn(0x7ff4700d0e70 0x7ff4700d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 -- 172.21.15.106:0/1123656976 >> 172.21.15.106:0/1123656976 conn(0x7ff47006c740 msgr2=0x7ff47006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 -- 172.21.15.106:0/1123656976 shutdown_connections 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.495+0000 7ff477200640 1 -- 172.21.15.106:0/1123656976 wait complete. 2024-04-26T21:46:09.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.496+0000 7ff477200640 1 Processor -- start 2024-04-26T21:46:09.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.496+0000 7ff477200640 1 -- start start 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 0x7ff4700cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4700d0e70 0x7ff4700ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 0x7ff4700cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff47006e320 con 0x7ff470146680 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700d17f0 con 0x7ff4700b3b20 2024-04-26T21:46:09.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.497+0000 7ff477200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff47006e600 con 0x7ff4700d0e70 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff475e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 0x7ff4700cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff475400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4700d0e70 0x7ff4700ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff475e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 0x7ff4700cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:39730/0 (socket says 172.21.15.106:39730) 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff475e00640 1 -- 172.21.15.106:0/3100302414 learned_addr learned my addr 172.21.15.106:0/3100302414 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 0x7ff4700cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 -- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4700d0e70 msgr2=0x7ff4700ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4700d0e70 0x7ff4700ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 -- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 msgr2=0x7ff4700cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 0x7ff4700cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.498+0000 7ff476800640 1 -- 172.21.15.106:0/3100302414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff470139c20 con 0x7ff470146680 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff476800640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 0x7ff4700cadf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff46c031000 tx=0x7ff46c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c041020 con 0x7ff470146680 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c030780 con 0x7ff470146680 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c037040 con 0x7ff470146680 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff470139f00 con 0x7ff470146680 2024-04-26T21:46:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.499+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47013a4c0 con 0x7ff470146680 2024-04-26T21:46:09.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.500+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4700693c0 con 0x7ff470146680 2024-04-26T21:46:09.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.501+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff46c044020 con 0x7ff470146680 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.502+0000 7ff466000640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 0x7ff45406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.502+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff46c0ac340 con 0x7ff470146680 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.503+0000 7ff475e00640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 0x7ff45406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.503+0000 7ff475e00640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 0x7ff45406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff468009fd0 tx=0x7ff46800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.507+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff46c04b030 con 0x7ff470146680 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.726+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff47006ce00 con 0x7ff470146680 2024-04-26T21:46:09.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.727+0000 7ff466000640 1 -- 172.21.15.106:0/3100302414 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7ff46c07a080 con 0x7ff470146680 2024-04-26T21:46:09.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:09.728 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:46:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 msgr2=0x7ff45406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 0x7ff45406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff468009fd0 tx=0x7ff46800b040 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 msgr2=0x7ff4700cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 0x7ff4700cadf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff46c031000 tx=0x7ff46c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 shutdown_connections 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470146680 0x7ff4700cadf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff45406c8f0 0x7ff45406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4700d0e70 0x7ff4700ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 --2- 172.21.15.106:0/3100302414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700b3b20 0x7ff4700cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 >> 172.21.15.106:0/3100302414 conn(0x7ff47006c740 msgr2=0x7ff47015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 shutdown_connections 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.730+0000 7ff477200640 1 -- 172.21.15.106:0/3100302414 wait complete. 2024-04-26T21:46:09.748 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:09.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.941+0000 7f66fa000640 1 Processor -- start 2024-04-26T21:46:09.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.941+0000 7f66fa000640 1 -- start start 2024-04-26T21:46:09.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.942+0000 7f66fa000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66f4146680 0x7f66f40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.942+0000 7f66fa000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66f40b4b30 0x7f66f40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.942+0000 7f66fa000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f4146a60 con 0x7f66f40d0e70 2024-04-26T21:46:09.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.942+0000 7f66fa000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f66f406e970 con 0x7f66f40b4b30 2024-04-26T21:46:09.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.942+0000 7f66fa000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66f40d17f0 con 0x7f66f4146680 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66f40b4b30 0x7f66f40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66f40b4b30 0x7f66f40d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35066/0 (socket says 172.21.15.106:35066) 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 -- 172.21.15.106:0/1975266673 learned_addr learned my addr 172.21.15.106:0/1975266673 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f8c00640 1 --2- 172.21.15.106:0/1975266673 >> v2:172.21.15.204:3300/0 conn(0x7f66f4146680 0x7f66f40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 -- 172.21.15.106:0/1975266673 >> v2:172.21.15.204:3300/0 conn(0x7f66f4146680 msgr2=0x7f66f40b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 --2- 172.21.15.106:0/1975266673 >> v2:172.21.15.204:3300/0 conn(0x7f66f4146680 0x7f66f40b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 -- 172.21.15.106:0/1975266673 >> v1:172.21.15.204:6789/0 conn(0x7f66f40d0e70 legacy=0x7f66f4160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.943+0000 7f66f9600640 1 -- 172.21.15.106:0/1975266673 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f406dd60 con 0x7f66f40b4b30 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.944+0000 7f66f9600640 1 --2- 172.21.15.106:0/1975266673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f40d12b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66f4074250 tx=0x7f66ec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdff2a3b7e627e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.944+0000 7f66f3400640 1 -- 172.21.15.106:0/1975266673 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec036070 con 0x7f66f40b4b30 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66f3400640 1 -- 172.21.15.106:0/1975266673 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ec031600 con 0x7f66f40b4b30 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66f3400640 1 -- 172.21.15.106:0/1975266673 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0319d0 con 0x7f66f40b4b30 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 -- 172.21.15.106:0/1975266673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 msgr2=0x7f66f40d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 --2- 172.21.15.106:0/1975266673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f40d12b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66f4074250 tx=0x7f66ec02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 -- 172.21.15.106:0/1975266673 shutdown_connections 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 --2- 172.21.15.106:0/1975266673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f40d12b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 --2- 172.21.15.106:0/1975266673 >> v2:172.21.15.204:3300/0 conn(0x7f66f4146680 0x7f66f40b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 -- 172.21.15.106:0/1975266673 >> 172.21.15.106:0/1975266673 conn(0x7f66f406c740 msgr2=0x7f66f406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 -- 172.21.15.106:0/1975266673 shutdown_connections 2024-04-26T21:46:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.945+0000 7f66fa000640 1 -- 172.21.15.106:0/1975266673 wait complete. 2024-04-26T21:46:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.947+0000 7f66fa000640 1 Processor -- start 2024-04-26T21:46:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.947+0000 7f66fa000640 1 -- start start 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66f40d0e70 0x7f66f41eff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66f4146680 0x7f66f413d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66f406e600 con 0x7f66f40b4b30 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66f406e320 con 0x7f66f4146680 2024-04-26T21:46:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66fa000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40cd6a0 con 0x7f66f40d0e70 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35080/0 (socket says 172.21.15.106:35080) 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f8c00640 1 -- 172.21.15.106:0/3449847395 learned_addr learned my addr 172.21.15.106:0/3449847395 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f8c00640 1 -- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66f40d0e70 msgr2=0x7f66f41eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f3e00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66f40d0e70 0x7f66f41eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.948+0000 7f66f8c00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66f40d0e70 0x7f66f41eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f8c00640 1 -- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66f4146680 msgr2=0x7f66f413d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f9600640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66f4146680 0x7f66f413d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f8c00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66f4146680 0x7f66f413d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f8c00640 1 -- 172.21.15.106:0/3449847395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f41f3a30 con 0x7f66f40b4b30 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f8c00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f66e800c820 tx=0x7f66e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8015070 con 0x7f66f40b4b30 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e8004020 con 0x7f66f40b4b30 2024-04-26T21:46:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f41f0470 con 0x7f66f40b4b30 2024-04-26T21:46:09.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80043f0 con 0x7f66f40b4b30 2024-04-26T21:46:09.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.949+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f413bed0 con 0x7f66f40b4b30 2024-04-26T21:46:09.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.950+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f40693c0 con 0x7f66f40b4b30 2024-04-26T21:46:09.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.952+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66e8007830 con 0x7f66f40b4b30 2024-04-26T21:46:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.953+0000 7f66f0c00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.953+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f66e808b870 con 0x7f66f40b4b30 2024-04-26T21:46:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.953+0000 7f66f3e00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.954+0000 7f66f3e00640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 0x7f66d406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f66f4146090 tx=0x7f66e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:09.958+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66f41f3a30 con 0x7f66f40b4b30 2024-04-26T21:46:10.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.156+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66f40b5e70 con 0x7f66f40b4b30 2024-04-26T21:46:10.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.159+0000 7f66f0c00640 1 -- 172.21.15.106:0/3449847395 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f66e8059d30 con 0x7f66f40b4b30 2024-04-26T21:46:10.160 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:10.161 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:10.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.161+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 msgr2=0x7f66d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.161+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 0x7f66d406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f66f4146090 tx=0x7f66e0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:10.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 msgr2=0x7f66f41f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f66e800c820 tx=0x7f66e800cce0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 shutdown_connections 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66f4146680 0x7f66f413d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66f40d0e70 0x7f66f41eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 --2- 172.21.15.106:0/3449847395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66f40b4b30 0x7f66f41f34f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 >> 172.21.15.106:0/3449847395 conn(0x7f66f406c740 msgr2=0x7f66f40b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:10.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 shutdown_connections 2024-04-26T21:46:10.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.162+0000 7f66fa000640 1 -- 172.21.15.106:0/3449847395 wait complete. 2024-04-26T21:46:10.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:10.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:10.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.367+0000 7fdd16e00640 1 Processor -- start 2024-04-26T21:46:10.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.367+0000 7fdd16e00640 1 -- start start 2024-04-26T21:46:10.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.369+0000 7fdd16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.369+0000 7fdd16e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.369+0000 7fdd16e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100d0170 con 0x7fdd10162cf0 2024-04-26T21:46:10.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.369+0000 7fdd16e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100d0450 con 0x7fdd1015fbd0 2024-04-26T21:46:10.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.369+0000 7fdd16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdd100d0730 con 0x7fdd1015f7f0 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:39758/0 (socket says 172.21.15.106:39758) 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:47730/0 (socket says 172.21.15.106:47730) 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15000640 1 -- 172.21.15.106:0/1355594447 learned_addr learned my addr 172.21.15.106:0/1355594447 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:10.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 -- 172.21.15.106:0/1355594447 >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 msgr2=0x7fdd100d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:10.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 --2- 172.21.15.106:0/1355594447 >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 -- 172.21.15.106:0/1355594447 >> v1:172.21.15.204:6789/0 conn(0x7fdd10162cf0 legacy=0x7fdd10163180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15a00640 1 -- 172.21.15.106:0/1355594447 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100d0a10 con 0x7fdd1015f7f0 2024-04-26T21:46:10.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.370+0000 7fdd15000640 1 --2- 172.21.15.106:0/1355594447 >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:10.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.372+0000 7fdd15a00640 1 --2- 172.21.15.106:0/1355594447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a66a3f6a9ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.372+0000 7fdd0fe00640 1 -- 172.21.15.106:0/1355594447 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031600 con 0x7fdd1015f7f0 2024-04-26T21:46:10.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.372+0000 7fdd0fe00640 1 -- 172.21.15.106:0/1355594447 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd000317a0 con 0x7fdd1015f7f0 2024-04-26T21:46:10.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.372+0000 7fdd0fe00640 1 -- 172.21.15.106:0/1355594447 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031b70 con 0x7fdd1015f7f0 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 -- 172.21.15.106:0/1355594447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 msgr2=0x7fdd100cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 --2- 172.21.15.106:0/1355594447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 -- 172.21.15.106:0/1355594447 shutdown_connections 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 --2- 172.21.15.106:0/1355594447 >> v2:172.21.15.204:3300/0 conn(0x7fdd1015fbd0 0x7fdd100d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 --2- 172.21.15.106:0/1355594447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015f7f0 0x7fdd100cfb90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 -- 172.21.15.106:0/1355594447 >> 172.21.15.106:0/1355594447 conn(0x7fdd100d4f80 msgr2=0x7fdd100d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:10.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 -- 172.21.15.106:0/1355594447 shutdown_connections 2024-04-26T21:46:10.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.373+0000 7fdd16e00640 1 -- 172.21.15.106:0/1355594447 wait complete. 2024-04-26T21:46:10.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.375+0000 7fdd16e00640 1 Processor -- start 2024-04-26T21:46:10.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.375+0000 7fdd16e00640 1 -- start start 2024-04-26T21:46:10.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd1015f7f0 0x7fdd100a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015fbd0 0x7fdd100b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10160050 con 0x7fdd10162cf0 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100d2810 con 0x7fdd1015f7f0 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1015f430 con 0x7fdd1015fbd0 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.376+0000 7fdd16400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35094/0 (socket says 172.21.15.106:35094) 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 -- 172.21.15.106:0/2915425865 learned_addr learned my addr 172.21.15.106:0/2915425865 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd15a00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd1015f7f0 0x7fdd100a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd15000640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015fbd0 0x7fdd100b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 -- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015fbd0 msgr2=0x7fdd100b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015fbd0 0x7fdd100b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 -- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd1015f7f0 msgr2=0x7fdd100a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd1015f7f0 0x7fdd100a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 -- 172.21.15.106:0/2915425865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100b8cf0 con 0x7fdd10162cf0 2024-04-26T21:46:10.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.377+0000 7fdd16400640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c390 tx=0x7fdd0800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.378+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.378+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd080052d0 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.378+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd100b8f90 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.378+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003810 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.378+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd100b9550 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.379+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd100693c0 con 0x7fdd10162cf0 2024-04-26T21:46:10.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.380+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd0801a070 con 0x7fdd10162cf0 2024-04-26T21:46:10.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.381+0000 7fdd0d600640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 0x7fdcf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.381+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd0808aec0 con 0x7fdd10162cf0 2024-04-26T21:46:10.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.381+0000 7fdd15a00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 0x7fdcf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.382+0000 7fdd15a00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 0x7fdcf006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.385+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd08059260 con 0x7fdd10162cf0 2024-04-26T21:46:10.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.581+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd1008ea90 con 0x7fdd10162cf0 2024-04-26T21:46:10.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.582+0000 7fdd0d600640 1 -- 172.21.15.106:0/2915425865 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdd08058b60 con 0x7fdd10162cf0 2024-04-26T21:46:10.583 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:10.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 msgr2=0x7fdcf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 0x7fdcf006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002f000 comp rx=0 tx=0).stop 2024-04-26T21:46:10.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 msgr2=0x7fdd100ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c390 tx=0x7fdd0800c850 comp rx=0 tx=0).stop 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 shutdown_connections 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd10162cf0 0x7fdd100ceb90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdcf006c8f0 0x7fdcf006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd1015fbd0 0x7fdd100b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 --2- 172.21.15.106:0/2915425865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd1015f7f0 0x7fdd100a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 >> 172.21.15.106:0/2915425865 conn(0x7fdd100d4f80 msgr2=0x7fdd1006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 shutdown_connections 2024-04-26T21:46:10.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.584+0000 7fdd16e00640 1 -- 172.21.15.106:0/2915425865 wait complete. 2024-04-26T21:46:10.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:10.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.798+0000 7fd4c5600640 1 Processor -- start 2024-04-26T21:46:10.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.798+0000 7fd4c5600640 1 -- start start 2024-04-26T21:46:10.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.799+0000 7fd4c5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4c006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.799+0000 7fd4c5600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4c0146280 0x7fd4c01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.799+0000 7fd4c5600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c00b4d20 con 0x7fd4c0160d60 2024-04-26T21:46:10.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.799+0000 7fd4c5600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c0146c30 con 0x7fd4c0146280 2024-04-26T21:46:10.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.799+0000 7fd4c5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c00b5000 con 0x7fd4c0161140 2024-04-26T21:46:10.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bfe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd4c0160d60 0x7fd4c00b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36394/0 (socket says 172.21.15.106:36394) 2024-04-26T21:46:10.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bfe00640 1 -- 172.21.15.106:0/2815076038 learned_addr learned my addr 172.21.15.106:0/2815076038 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 290384855 0 0) 0x7fd4c00b4d20 con 0x7fd4c0160d60 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4b7e00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4c006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4a4003520 con 0x7fd4c0160d60 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/2815076038 >> v2:172.21.15.204:3300/0 conn(0x7fd4c0146280 0x7fd4c01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2626819206 0 0) 0x7fd4a4003520 con 0x7fd4c0160d60 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 msgr2=0x7fd4c006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4c006dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 >> v2:172.21.15.204:3300/0 conn(0x7fd4c0146280 msgr2=0x7fd4c01466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 --2- 172.21.15.106:0/2815076038 >> v2:172.21.15.204:3300/0 conn(0x7fd4c0146280 0x7fd4c01466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c00b52e0 con 0x7fd4c0160d60 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fd4b0001670 con 0x7fd4c0160d60 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 >> v1:172.21.15.106:6789/0 conn(0x7fd4c0160d60 legacy=0x7fd4c00b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.800+0000 7fd4bf400640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a4003a20 0x7fd4a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bf400640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bf400640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4a4004d40 0x7fd4a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0146c30 con 0x7fd4a4004d40 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c00b5000 con 0x7fd4a4003a20 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a400ba70 con 0x7fd4c0161140 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4a4004d40 0x7fd4a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4bfe00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a4003a20 0x7fd4a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4b7e00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 -- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 msgr2=0x7fd4a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 -- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a4003a20 msgr2=0x7fd4a4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:10.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a4003a20 0x7fd4a4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4c4c00640 1 -- 172.21.15.106:0/2815076038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a4003520 con 0x7fd4a4004d40 2024-04-26T21:46:10.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.801+0000 7fd4b7e00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c0161140 0x7fd4a4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:10.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.802+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4a4004d40 0x7fd4a40051c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8007360 tx=0x7fd4a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f66ddf43751aea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8009be0 con 0x7fd4a4004d40 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b802e640 con 0x7fd4a4004d40 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4bf400640 1 -- 172.21.15.106:0/2815076038 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b802ea10 con 0x7fd4a4004d40 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 -- 172.21.15.106:0/2815076038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4a4004d40 msgr2=0x7fd4a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 --2- 172.21.15.106:0/2815076038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4a4004d40 0x7fd4a40051c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8007360 tx=0x7fd4a4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:10.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c4c00640 1 -- 172.21.15.106:0/2815076038 reap_dead start 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 -- 172.21.15.106:0/2815076038 shutdown_connections 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 -- 172.21.15.106:0/2815076038 >> 172.21.15.106:0/2815076038 conn(0x7fd4c00b2900 msgr2=0x7fd4c00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 -- 172.21.15.106:0/2815076038 shutdown_connections 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.803+0000 7fd4c5600640 1 -- 172.21.15.106:0/2815076038 wait complete. 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.805+0000 7fd4c5600640 1 Processor -- start 2024-04-26T21:46:10.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.805+0000 7fd4c5600640 1 -- start start 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a4004d40 0x7fd4c0159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c013f010 0x7fd4c013a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0161770 con 0x7fd4c013dd10 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c00cf6f0 con 0x7fd4c013f010 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c01618f0 con 0x7fd4a4004d40 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35126/0 (socket says 172.21.15.106:35126) 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.806+0000 7fd4c4c00640 1 -- 172.21.15.106:0/1728553029 learned_addr learned my addr 172.21.15.106:0/1728553029 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4b7e00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c013f010 0x7fd4c013a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 -- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a4004d40 msgr2=0x7fd4c0159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a4004d40 0x7fd4c0159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 -- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c013f010 msgr2=0x7fd4c013a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c013f010 0x7fd4c013a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 -- 172.21.15.106:0/1728553029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c013f510 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c4c00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd4b802b0c0 tx=0x7fd4b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8009be0 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b802e440 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c0158520 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8038890 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.807+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c013ad30 con 0x7fd4c013dd10 2024-04-26T21:46:10.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.809+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c00693c0 con 0x7fd4c013dd10 2024-04-26T21:46:10.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.809+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4b8038a30 con 0x7fd4c013dd10 2024-04-26T21:46:10.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.810+0000 7fd4bcc00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 0x7fd49c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:10.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.810+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4b80ac330 con 0x7fd4c013dd10 2024-04-26T21:46:10.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.811+0000 7fd4bfe00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 0x7fd49c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:10.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.811+0000 7fd4bfe00640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 0x7fd49c06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000f60 tx=0x7fd4b0000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:10.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:10.815+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4b8032030 con 0x7fd4c013dd10 2024-04-26T21:46:11.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.006+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4c00d14b0 con 0x7fd4c013dd10 2024-04-26T21:46:11.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.006+0000 7fd4bcc00640 1 -- 172.21.15.106:0/1728553029 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd4b807a750 con 0x7fd4c013dd10 2024-04-26T21:46:11.007 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.009+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 msgr2=0x7fd49c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.009+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 0x7fd49c06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000f60 tx=0x7fd4b0000ed0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.009+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 msgr2=0x7fd4c0140cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.009+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd4b802b0c0 tx=0x7fd4b8004ab0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 shutdown_connections 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c013dd10 0x7fd4c0140cf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c013f010 0x7fd4c013a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd49c06c920 0x7fd49c06ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 --2- 172.21.15.106:0/1728553029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a4004d40 0x7fd4c0159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 >> 172.21.15.106:0/1728553029 conn(0x7fd4c00b2900 msgr2=0x7fd4c00b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 shutdown_connections 2024-04-26T21:46:11.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.010+0000 7fd4c5600640 1 -- 172.21.15.106:0/1728553029 wait complete. 2024-04-26T21:46:11.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:11.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.216+0000 7f1c46200640 1 Processor -- start 2024-04-26T21:46:11.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.217+0000 7f1c46200640 1 -- start start 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c46200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d19c0 0x7f1c400d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c46200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c400d1da0 0x7f1c40162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c46200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400d3830 con 0x7f1c400b6170 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c46200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1c400d40d0 con 0x7f1c400d1da0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c46200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1c400d3b10 con 0x7f1c400d19c0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c3ea00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1c400b6170 0x7f1c400c8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36396/0 (socket says 172.21.15.106:36396) 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c3ea00640 1 -- 172.21.15.106:0/28383119 learned_addr learned my addr 172.21.15.106:0/28383119 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.218+0000 7f1c3fe00640 1 --2- 172.21.15.106:0/28383119 >> v2:172.21.15.106:3300/0 conn(0x7f1c400d1da0 0x7f1c40162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3f400640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d19c0 0x7f1c400d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3e000640 1 -- 172.21.15.106:0/28383119 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606056437 0 0) 0x7f1c400d3830 con 0x7f1c400b6170 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3e000640 1 -- 172.21.15.106:0/28383119 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c24003520 con 0x7f1c400b6170 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3fe00640 1 -- 172.21.15.106:0/28383119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d19c0 msgr2=0x7f1c400d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3fe00640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d19c0 0x7f1c400d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3fe00640 1 -- 172.21.15.106:0/28383119 >> v1:172.21.15.106:6789/0 conn(0x7f1c400b6170 legacy=0x7f1c400c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.219+0000 7f1c3fe00640 1 -- 172.21.15.106:0/28383119 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400d3df0 con 0x7f1c400d1da0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.220+0000 7f1c3fe00640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400d1da0 0x7f1c40162640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009d90 tx=0x7f1c3802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a973245b66e60ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.220+0000 7f1c3e000640 1 -- 172.21.15.106:0/28383119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38031600 con 0x7f1c400d1da0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.220+0000 7f1c3e000640 1 -- 172.21.15.106:0/28383119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c380317a0 con 0x7f1c400d1da0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.220+0000 7f1c3e000640 1 -- 172.21.15.106:0/28383119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38031b50 con 0x7f1c400d1da0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 -- 172.21.15.106:0/28383119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400d1da0 msgr2=0x7f1c40162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400d1da0 0x7f1c40162640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009d90 tx=0x7f1c3802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 -- 172.21.15.106:0/28383119 shutdown_connections 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400d1da0 0x7f1c40162640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 --2- 172.21.15.106:0/28383119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d19c0 0x7f1c400d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 -- 172.21.15.106:0/28383119 >> 172.21.15.106:0/28383119 conn(0x7f1c4009e460 msgr2=0x7f1c400d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 -- 172.21.15.106:0/28383119 shutdown_connections 2024-04-26T21:46:11.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.221+0000 7f1c46200640 1 -- 172.21.15.106:0/28383119 wait complete. 2024-04-26T21:46:11.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.222+0000 7f1c46200640 1 Processor -- start 2024-04-26T21:46:11.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 -- start start 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c400d19c0 0x7f1c4015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d1da0 0x7f1c4015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c40162b80 con 0x7f1c400b6170 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400cfc30 con 0x7f1c400d19c0 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.223+0000 7f1c46200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1c40162d00 con 0x7f1c400d1da0 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35138/0 (socket says 172.21.15.106:35138) 2024-04-26T21:46:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 -- 172.21.15.106:0/2669887050 learned_addr learned my addr 172.21.15.106:0/2669887050 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 -- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d1da0 msgr2=0x7f1c4015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d1da0 0x7f1c4015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 -- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c400d19c0 msgr2=0x7f1c4015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c400d19c0 0x7f1c4015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 -- 172.21.15.106:0/2669887050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c401563d0 con 0x7f1c400b6170 2024-04-26T21:46:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c3f400640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000d800 tx=0x7f1c3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.224+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000b6c0 con 0x7f1c400b6170 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.225+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c30019070 con 0x7f1c400b6170 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.225+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000ba80 con 0x7f1c400b6170 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.225+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c401566b0 con 0x7f1c400b6170 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.225+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c40159370 con 0x7f1c400b6170 2024-04-26T21:46:11.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.226+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c40069650 con 0x7f1c400b6170 2024-04-26T21:46:11.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.227+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c3000bc20 con 0x7f1c400b6170 2024-04-26T21:46:11.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.228+0000 7f1c37400640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 0x7f1c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.228+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1c3008c050 con 0x7f1c400b6170 2024-04-26T21:46:11.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.228+0000 7f1c3ea00640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 0x7f1c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.229+0000 7f1c3ea00640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 0x7f1c1c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000f60 tx=0x7f1c280078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.232+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c3001c360 con 0x7f1c400b6170 2024-04-26T21:46:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.417+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c400d26d0 con 0x7f1c400b6170 2024-04-26T21:46:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.418+0000 7f1c37400640 1 -- 172.21.15.106:0/2669887050 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1c3005a470 con 0x7f1c400b6170 2024-04-26T21:46:11.419 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 msgr2=0x7f1c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 0x7f1c1c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000f60 tx=0x7f1c280078f0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 msgr2=0x7f1c40158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000d800 tx=0x7f1c3000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 shutdown_connections 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1c1c06c8f0 0x7f1c1c06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c400d1da0 0x7f1c4015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c400d19c0 0x7f1c4015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 --2- 172.21.15.106:0/2669887050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c400b6170 0x7f1c40158e30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 >> 172.21.15.106:0/2669887050 conn(0x7f1c4009e460 msgr2=0x7f1c40160af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.420+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 shutdown_connections 2024-04-26T21:46:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.421+0000 7f1c46200640 1 -- 172.21.15.106:0/2669887050 wait complete. 2024-04-26T21:46:11.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.624+0000 7f6f75c00640 1 Processor -- start 2024-04-26T21:46:11.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.624+0000 7f6f75c00640 1 -- start start 2024-04-26T21:46:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700d3680 0x7f6f700d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f75c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f700c9520 0x7f6f700c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f75c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f7015f800 con 0x7f6f7015f420 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f75c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6f700d2c70 con 0x7f6f700c9520 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f75c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f700d3480 con 0x7f6f700d3680 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f6f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f700c9520 0x7f6f700c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f6f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6f700c9520 0x7f6f700c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35146/0 (socket says 172.21.15.106:35146) 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f6f400640 1 -- 172.21.15.106:0/1974419867 learned_addr learned my addr 172.21.15.106:0/1974419867 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.626+0000 7f6f66000640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700d3680 0x7f6f700d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.627+0000 7f6f6f400640 1 -- 172.21.15.106:0/1974419867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700d3680 msgr2=0x7f6f700d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.627+0000 7f6f6f400640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700d3680 0x7f6f700d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.627+0000 7f6f6f400640 1 -- 172.21.15.106:0/1974419867 >> v1:172.21.15.204:6789/0 conn(0x7f6f7015f420 legacy=0x7f6f700d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.627+0000 7f6f6f400640 1 -- 172.21.15.106:0/1974419867 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f700d4020 con 0x7f6f700c9520 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.628+0000 7f6f6f400640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700c9520 0x7f6f700c99b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009e50 tx=0x7f6f6802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b4a98fcf00cc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.628+0000 7f6f6e000640 1 -- 172.21.15.106:0/1974419867 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68036070 con 0x7f6f700c9520 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.628+0000 7f6f6e000640 1 -- 172.21.15.106:0/1974419867 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f68031600 con 0x7f6f700c9520 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.628+0000 7f6f6e000640 1 -- 172.21.15.106:0/1974419867 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680319d0 con 0x7f6f700c9520 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 -- 172.21.15.106:0/1974419867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700c9520 msgr2=0x7f6f700c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700c9520 0x7f6f700c99b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009e50 tx=0x7f6f6802a630 comp rx=0 tx=0).stop 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 -- 172.21.15.106:0/1974419867 shutdown_connections 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700c9520 0x7f6f700c99b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 --2- 172.21.15.106:0/1974419867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700d3680 0x7f6f700d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 -- 172.21.15.106:0/1974419867 >> 172.21.15.106:0/1974419867 conn(0x7f6f7009e3e0 msgr2=0x7f6f700d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 -- 172.21.15.106:0/1974419867 shutdown_connections 2024-04-26T21:46:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.629+0000 7f6f75c00640 1 -- 172.21.15.106:0/1974419867 wait complete. 2024-04-26T21:46:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.631+0000 7f6f75c00640 1 Processor -- start 2024-04-26T21:46:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.631+0000 7f6f75c00640 1 -- start start 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700c9520 0x7f6f700b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f7015f420 0x7f6f70161950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f700c8d20 con 0x7f6f700d3680 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f700d31a0 con 0x7f6f7015f420 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f75c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f700c9000 con 0x7f6f700c9520 2024-04-26T21:46:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35162/0 (socket says 172.21.15.106:35162) 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 -- 172.21.15.106:0/3094848445 learned_addr learned my addr 172.21.15.106:0/3094848445 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f6ea00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700c9520 0x7f6f700b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 -- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700c9520 msgr2=0x7f6f700b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700c9520 0x7f6f700b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 -- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f7015f420 msgr2=0x7f6f70161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f7015f420 0x7f6f70161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.632+0000 7f6f66000640 1 -- 172.21.15.106:0/3094848445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f700cc380 con 0x7f6f700d3680 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f66000640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d800 tx=0x7f6f5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800b6c0 con 0x7f6f700d3680 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f58019070 con 0x7f6f700d3680 2024-04-26T21:46:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800ba80 con 0x7f6f700d3680 2024-04-26T21:46:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f700cedc0 con 0x7f6f700d3680 2024-04-26T21:46:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.633+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f70161430 con 0x7f6f700d3680 2024-04-26T21:46:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.634+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f70069540 con 0x7f6f700d3680 2024-04-26T21:46:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.635+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f5800bc20 con 0x7f6f700d3680 2024-04-26T21:46:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.636+0000 7f6f67400640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 0x7f6f4c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.636+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f5808be40 con 0x7f6f700d3680 2024-04-26T21:46:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.636+0000 7f6f6ea00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 0x7f6f4c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.637+0000 7f6f6ea00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 0x7f6f4c06ee30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000a750 tx=0x7f6f60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.640+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f5805a3e0 con 0x7f6f700d3680 2024-04-26T21:46:11.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.883+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7015ebd0 con 0x7f6f700d3680 2024-04-26T21:46:11.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.884+0000 7f6f67400640 1 -- 172.21.15.106:0/3094848445 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f6f58059ce0 con 0x7f6f700d3680 2024-04-26T21:46:11.885 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:11.885 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:46:11.886 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:46:11.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.886+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 msgr2=0x7f6f4c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.886+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 0x7f6f4c06ee30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000a750 tx=0x7f6f60008040 comp rx=0 tx=0).stop 2024-04-26T21:46:11.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 msgr2=0x7f6f701536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d800 tx=0x7f6f5800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 shutdown_connections 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f7015f420 0x7f6f70161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f700d3680 0x7f6f701536b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6f4c06c970 0x7f6f4c06ee30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 --2- 172.21.15.106:0/3094848445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f700c9520 0x7f6f700b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 >> 172.21.15.106:0/3094848445 conn(0x7f6f7009e3e0 msgr2=0x7f6f7015b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 shutdown_connections 2024-04-26T21:46:11.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:11.887+0000 7f6f75c00640 1 -- 172.21.15.106:0/3094848445 wait complete. 2024-04-26T21:46:11.907 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:12.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.102+0000 7f28f2c00640 1 Processor -- start 2024-04-26T21:46:12.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.103+0000 7f28f2c00640 1 -- start start 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f2c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28ec15f5f0 0x7f28ec0d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f2c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28ec0d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f2c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ec0d5e20 con 0x7f28ec0d2e90 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f2c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f28ec0d5bd0 con 0x7f28ec15f5f0 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f2c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec0d11e0 con 0x7f28ec0b5c40 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28eb400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f28ec0d2e90 0x7f28ec0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36406/0 (socket says 172.21.15.106:36406) 2024-04-26T21:46:12.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28eb400640 1 -- 172.21.15.106:0/135640716 learned_addr learned my addr 172.21.15.106:0/135640716 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28ebe00640 1 --2- 172.21.15.106:0/135640716 >> v2:172.21.15.204:3300/0 conn(0x7f28ec15f5f0 0x7f28ec0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470034927 0 0) 0x7f28ec0d5e20 con 0x7f28ec0d2e90 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28f0c00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28ec0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.104+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28d0003520 con 0x7f28ec0d2e90 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3779459089 0 0) 0x7f28d0003520 con 0x7f28ec0d2e90 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 msgr2=0x7f28ec0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28ec0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 >> v2:172.21.15.204:3300/0 conn(0x7f28ec15f5f0 msgr2=0x7f28ec0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 --2- 172.21.15.106:0/135640716 >> v2:172.21.15.204:3300/0 conn(0x7f28ec15f5f0 0x7f28ec0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0d14c0 con 0x7f28ec0d2e90 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f28d8001670 con 0x7f28ec0d2e90 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 >> v1:172.21.15.106:6789/0 conn(0x7f28ec0d2e90 legacy=0x7f28ec0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d00039b0 0x7f28d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28d0004800 0x7f28d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0d5bd0 con 0x7f28d0004800 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0d11e0 con 0x7f28d00039b0 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28d000ba70 con 0x7f28ec0b5c40 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.105+0000 7f28eb400640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28d0004800 0x7f28d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28f0c00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28ebe00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d00039b0 0x7f28d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28eb400640 1 -- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 msgr2=0x7f28d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28eb400640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28eb400640 1 -- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d00039b0 msgr2=0x7f28d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28eb400640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d00039b0 0x7f28d0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28eb400640 1 -- 172.21.15.106:0/135640716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d0003520 con 0x7f28d0004800 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.106+0000 7f28f0c00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec0b5c40 0x7f28d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.107+0000 7f28eb400640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28d0004800 0x7f28d0004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007fd0 tx=0x7f28d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6ac7e25b291af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.107+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8007d50 con 0x7f28d0004800 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.107+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d8005af0 con 0x7f28d0004800 2024-04-26T21:46:12.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.107+0000 7f28eaa00640 1 -- 172.21.15.106:0/135640716 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8007070 con 0x7f28d0004800 2024-04-26T21:46:12.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 -- 172.21.15.106:0/135640716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28d0004800 msgr2=0x7f28d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 --2- 172.21.15.106:0/135640716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28d0004800 0x7f28d0004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007fd0 tx=0x7f28d0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:12.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f0c00640 1 -- 172.21.15.106:0/135640716 reap_dead start 2024-04-26T21:46:12.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 -- 172.21.15.106:0/135640716 shutdown_connections 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 -- 172.21.15.106:0/135640716 >> 172.21.15.106:0/135640716 conn(0x7f28ec0d50b0 msgr2=0x7f28ec0d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 -- 172.21.15.106:0/135640716 shutdown_connections 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.108+0000 7f28f2c00640 1 -- 172.21.15.106:0/135640716 wait complete. 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.110+0000 7f28f2c00640 1 Processor -- start 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.110+0000 7f28f2c00640 1 -- start start 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d0004800 0x7f28ec160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec150490 0x7f28ec0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0b4100 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0b3f80 con 0x7f28d0004800 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f2c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec0b4280 con 0x7f28ec150490 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35178/0 (socket says 172.21.15.106:35178) 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 -- 172.21.15.106:0/2395511784 learned_addr learned my addr 172.21.15.106:0/2395511784 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28eb400640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec150490 0x7f28ec0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 -- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec150490 msgr2=0x7f28ec0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec150490 0x7f28ec0cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 -- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d0004800 msgr2=0x7f28ec160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d0004800 0x7f28ec160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.111+0000 7f28f0c00640 1 -- 172.21.15.106:0/2395511784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec161440 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28f0c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28ec08d0e0 tx=0x7f28e400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400f040 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4019070 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec161160 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4013870 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.112+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec161a00 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.113+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec069540 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.114+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28e4013a10 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.115+0000 7f28e3e00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 0x7f28cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.116+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28e408b2f0 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.116+0000 7f28ebe00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 0x7f28cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.116+0000 7f28ebe00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 0x7f28cc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0094c0 tx=0x7f28dc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.120+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28e4059720 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.318+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28ec0d3470 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.321+0000 7f28e3e00640 1 -- 172.21.15.106:0/2395511784 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f28e4059020 con 0x7f28ec0cc6c0 2024-04-26T21:46:12.323 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:12.323 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 msgr2=0x7f28cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 0x7f28cc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0094c0 tx=0x7f28dc00b040 comp rx=0 tx=0).stop 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 msgr2=0x7f28ec162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28ec08d0e0 tx=0x7f28e400e830 comp rx=0 tx=0).stop 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 shutdown_connections 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0cc6c0 0x7f28ec162200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28cc06c8f0 0x7f28cc06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28ec150490 0x7f28ec0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 --2- 172.21.15.106:0/2395511784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28d0004800 0x7f28ec160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 >> 172.21.15.106:0/2395511784 conn(0x7f28ec0d50b0 msgr2=0x7f28ec0b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:12.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 shutdown_connections 2024-04-26T21:46:12.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.324+0000 7f28f2c00640 1 -- 172.21.15.106:0/2395511784 wait complete. 2024-04-26T21:46:12.343 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:12.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.534+0000 7faaf9c00640 1 Processor -- start 2024-04-26T21:46:12.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.534+0000 7faaf9c00640 1 -- start start 2024-04-26T21:46:12.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.535+0000 7faaf9c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 0x7faaf40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf40d0e70 0x7faaf40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faaf4160930 0x7faaf4160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7faaf406e970 con 0x7faaf4160930 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faaf40d17f0 con 0x7faaf4146680 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faaf4146a60 con 0x7faaf40d0e70 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faaf4160930 0x7faaf4160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faaf4160930 0x7faaf4160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35182/0 (socket says 172.21.15.106:35182) 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 -- 172.21.15.106:0/4068142566 learned_addr learned my addr 172.21.15.106:0/4068142566 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 -- 172.21.15.106:0/4068142566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf40d0e70 msgr2=0x7faaf40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaeb400640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf40d0e70 0x7faaf40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf3e00640 1 --2- 172.21.15.106:0/4068142566 >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 0x7faaf40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf40d0e70 0x7faaf40d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 -- 172.21.15.106:0/4068142566 >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 msgr2=0x7faaf40b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:12.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 --2- 172.21.15.106:0/4068142566 >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 0x7faaf40b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf9200640 1 -- 172.21.15.106:0/4068142566 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf406dd60 con 0x7faaf4160930 2024-04-26T21:46:12.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.536+0000 7faaf3e00640 1 --2- 172.21.15.106:0/4068142566 >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 0x7faaf40b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.537+0000 7faaf9200640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf4160930 0x7faaf4160dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaf4074250 tx=0x7faaec02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9625bbf6f0315 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf3400640 1 -- 172.21.15.106:0/4068142566 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec037070 con 0x7faaf4160930 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf3400640 1 -- 172.21.15.106:0/4068142566 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec032440 con 0x7faaf4160930 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf3400640 1 -- 172.21.15.106:0/4068142566 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec032810 con 0x7faaf4160930 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 -- 172.21.15.106:0/4068142566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf4160930 msgr2=0x7faaf4160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf4160930 0x7faaf4160dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaf4074250 tx=0x7faaec02a1d0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 -- 172.21.15.106:0/4068142566 shutdown_connections 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf4160930 0x7faaf4160dc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 --2- 172.21.15.106:0/4068142566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf40d0e70 0x7faaf40d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 --2- 172.21.15.106:0/4068142566 >> v2:172.21.15.204:3300/0 conn(0x7faaf4146680 0x7faaf40b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 -- 172.21.15.106:0/4068142566 >> 172.21.15.106:0/4068142566 conn(0x7faaf406c740 msgr2=0x7faaf406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 -- 172.21.15.106:0/4068142566 shutdown_connections 2024-04-26T21:46:12.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.538+0000 7faaf9c00640 1 -- 172.21.15.106:0/4068142566 wait complete. 2024-04-26T21:46:12.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.540+0000 7faaf9c00640 1 Processor -- start 2024-04-26T21:46:12.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.540+0000 7faaf9c00640 1 -- start start 2024-04-26T21:46:12.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf4146680 0x7faaf4164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf4160930 0x7faaf40b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faaf406e600 con 0x7faaf40d0e70 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faaf406e320 con 0x7faaf4146680 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf9c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faaf40cd6a0 con 0x7faaf4160930 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35184/0 (socket says 172.21.15.106:35184) 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 -- 172.21.15.106:0/2112435411 learned_addr learned my addr 172.21.15.106:0/2112435411 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 -- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf4160930 msgr2=0x7faaf40b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf4160930 0x7faaf40b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 -- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf4146680 msgr2=0x7faaf4164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf4146680 0x7faaf4164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.541+0000 7faaf3e00640 1 -- 172.21.15.106:0/2112435411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4165220 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf3e00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faae400c7f0 tx=0x7faae400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4015070 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4007a30 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4007e00 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41621f0 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.542+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41627b0 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.543+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf41629b0 con 0x7faaf40d0e70 2024-04-26T21:46:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.544+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faae4018070 con 0x7faaf40d0e70 2024-04-26T21:46:12.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.545+0000 7faaf0c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:12.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.545+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faae408bb70 con 0x7faaf40d0e70 2024-04-26T21:46:12.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.545+0000 7faaeb400640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:12.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.546+0000 7faaeb400640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faadc007130 tx=0x7faadc007310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:12.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.549+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faaf41629b0 con 0x7faaf40d0e70 2024-04-26T21:46:12.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.775+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf40693c0 con 0x7faaf40d0e70 2024-04-26T21:46:12.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.776+0000 7faaf0c00640 1 -- 172.21.15.106:0/2112435411 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7faae4059f90 con 0x7faaf40d0e70 2024-04-26T21:46:12.777 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:12.778 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:46:12.778 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.778+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 msgr2=0x7faad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.778+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faadc007130 tx=0x7faadc007310 comp rx=0 tx=0).stop 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 msgr2=0x7faaf415b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faae400c7f0 tx=0x7faae400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 shutdown_connections 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaf4160930 0x7faaf40b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf4146680 0x7faaf4164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 --2- 172.21.15.106:0/2112435411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaf40d0e70 0x7faaf415b180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:12.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 >> 172.21.15.106:0/2112435411 conn(0x7faaf406c740 msgr2=0x7faaf40b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:12.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 shutdown_connections 2024-04-26T21:46:12.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:12.779+0000 7faaf9c00640 1 -- 172.21.15.106:0/2112435411 wait complete. 2024-04-26T21:46:13.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:45:56.102085+0000', 'modified': '2024-04-26T21:45:59.159592+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34166}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34166': {'gid': 34166, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/4043819607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 4043819607}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 4043819607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34166, 'qdb_cluster': [34166]} 2024-04-26T21:46:13.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:46:13.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T21:46:13.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.397+0000 7fa6e0a00640 1 Processor -- start 2024-04-26T21:46:13.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.397+0000 7fa6e0a00640 1 -- start start 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6e0a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6dc0b5d40 0x7fa6dc0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6e0a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc0d1790 con 0x7fa6dc0b5660 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6e0a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc0727f0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b4a10 con 0x7fa6dc0d0e70 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6d9e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa6dc0b5660 0x7fa6dc0ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36416/0 (socket says 172.21.15.106:36416) 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6d9e00640 1 -- 172.21.15.106:0/1317997825 learned_addr learned my addr 172.21.15.106:0/1317997825 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6db200640 1 --2- 172.21.15.106:0/1317997825 >> v2:172.21.15.106:3300/0 conn(0x7fa6dc0b5d40 0x7fa6dc0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6da800640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.399+0000 7fa6d9400640 1 -- 172.21.15.106:0/1317997825 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429183265 0 0) 0x7fa6dc0d1790 con 0x7fa6dc0b5660 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.400+0000 7fa6d9400640 1 -- 172.21.15.106:0/1317997825 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6c0003520 con 0x7fa6dc0b5660 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.400+0000 7fa6db200640 1 -- 172.21.15.106:0/1317997825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 msgr2=0x7fa6dc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.400+0000 7fa6db200640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.400+0000 7fa6db200640 1 -- 172.21.15.106:0/1317997825 >> v1:172.21.15.106:6789/0 conn(0x7fa6dc0b5660 legacy=0x7fa6dc0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.400+0000 7fa6db200640 1 -- 172.21.15.106:0/1317997825 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc0b4cf0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.401+0000 7fa6db200640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc0aefd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009d90 tx=0x7fa6d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f7632bd370cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.401+0000 7fa6d9400640 1 -- 172.21.15.106:0/1317997825 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4031600 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.401+0000 7fa6d9400640 1 -- 172.21.15.106:0/1317997825 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d40317a0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.401+0000 7fa6d9400640 1 -- 172.21.15.106:0/1317997825 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4031b70 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 -- 172.21.15.106:0/1317997825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 msgr2=0x7fa6dc0aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc0aefd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009d90 tx=0x7fa6d402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 -- 172.21.15.106:0/1317997825 shutdown_connections 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc0aefd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/1317997825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 -- 172.21.15.106:0/1317997825 >> 172.21.15.106:0/1317997825 conn(0x7fa6dc06c740 msgr2=0x7fa6dc06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 -- 172.21.15.106:0/1317997825 shutdown_connections 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.402+0000 7fa6e0a00640 1 -- 172.21.15.106:0/1317997825 wait complete. 2024-04-26T21:46:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.403+0000 7fa6e0a00640 1 Processor -- start 2024-04-26T21:46:13.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.404+0000 7fa6e0a00640 1 -- start start 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.404+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc0b5660 0x7fa6dc160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.404+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b05b0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b0430 con 0x7fa6dc0b5660 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc15f1a0 con 0x7fa6dc0d0e70 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35210/0 (socket says 172.21.15.106:35210) 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2982014766 learned_addr learned my addr 172.21.15.106:0/2982014766 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 msgr2=0x7fa6dc151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc0b5660 msgr2=0x7fa6dc160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc0b5660 0x7fa6dc160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2982014766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc160c80 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.405+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c400 tx=0x7fa6c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.406+0000 7fa6d2a00640 1 -- 172.21.15.106:0/2982014766 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4017070 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.406+0000 7fa6d2a00640 1 -- 172.21.15.106:0/2982014766 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c4005d00 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.406+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc15f6a0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.406+0000 7fa6d2a00640 1 -- 172.21.15.106:0/2982014766 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007310 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.406+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc15fc60 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.408+0000 7fa6d2a00640 1 -- 172.21.15.106:0/2982014766 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6c401a070 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.409+0000 7fa6d2a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 0x7fa6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.410+0000 7fa6d2a00640 1 -- 172.21.15.106:0/2982014766 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c408afe0 con 0x7fa6dc0b5d40 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.410+0000 7fa6da800640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 0x7fa6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.410+0000 7fa6e426e740 1 Processor -- start 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.410+0000 7fa6e426e740 1 -- start start 2024-04-26T21:46:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.410+0000 7fa6da800640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 0x7fa6b806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006fd0 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 0x558f3b612910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558f3b574600 con 0x558f3b64ce80 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3b511100 con 0x558f3b63b270 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558f3b5705c0 con 0x558f3b6107b0 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35212/0 (socket says 172.21.15.106:35212) 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2522068836 learned_addr learned my addr 172.21.15.106:0/2522068836 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.411+0000 7fa6da800640 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 0x558f3b612910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 msgr2=0x558f3b612910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 0x558f3b612910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 msgr2=0x558f3b63b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 -- 172.21.15.106:0/2522068836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3b2862d0 con 0x558f3b64ce80 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6da800640 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 0x558f3b612910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cfd0 tx=0x7fa6c4058e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242b7a503428f386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6cb000640 1 -- 172.21.15.106:0/2522068836 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4057020 con 0x558f3b64ce80 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6cb000640 1 -- 172.21.15.106:0/2522068836 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c40711c0 con 0x558f3b64ce80 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.412+0000 7fa6cb000640 1 -- 172.21.15.106:0/2522068836 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40074b0 con 0x558f3b64ce80 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- 172.21.15.106:0/2522068836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 msgr2=0x558f3b610b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cfd0 tx=0x7fa6c4058e60 comp rx=0 tx=0).stop 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- 172.21.15.106:0/2522068836 shutdown_connections 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b64ce80 0x558f3b610b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 --2- 172.21.15.106:0/2522068836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b6107b0 0x558f3b612910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- 172.21.15.106:0/2522068836 >> 172.21.15.106:0/2522068836 conn(0x558f3b5a4d10 msgr2=0x558f3b656270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- 172.21.15.106:0/2522068836 shutdown_connections 2024-04-26T21:46:13.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- 172.21.15.106:0/2522068836 wait complete. 2024-04-26T21:46:13.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.413+0000 7fa6e426e740 1 -- start start 2024-04-26T21:46:13.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.414+0000 7fa6e426e740 1 Processor -- start 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b665db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558f3b4fbf70 con 0x558f3b6107b0 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3b638dc0 con 0x558f3b63b270 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558f3b63e2a0 con 0x558f3b64ce80 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6e426e740 10 client.0 _ll_register_callbacks cb 0x558f3b56a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35214/0 (socket says 172.21.15.106:35214) 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 -- 172.21.15.106:0/289040912 learned_addr learned my addr 172.21.15.106:0/289040912 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6db200640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b665db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 -- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 msgr2=0x558f3b665db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b665db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 -- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 msgr2=0x558f3b63dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.415+0000 7fa6da800640 1 -- 172.21.15.106:0/289040912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3b3506b0 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6da800640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00dea0 tx=0x7fa6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19362c10026b640 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01a050 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6cc0125f0 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c40711c0 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc014e20 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6e426e740 10 client.4668 fetch_fsmap learned FSMap version 7 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558f3b6667c0 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c40711c0 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fa6cc02f080 con 0x558f3b6107b0 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6e426e740 10 client.4668 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T21:46:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.416+0000 7fa6e426e740 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.417+0000 7fa6e426e740 20 client.4668 populate_metadata read hostname 'smithi106' 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.417+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b653120 0x558f3b6738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.417+0000 7fa6e426e740 4 client.4668 mds_command: new command op to 34166 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.417+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 --> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3b582210 con 0x558f3b653120 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.417+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b653120 0x558f3b6738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.418+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b653120 0x558f3b6738b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34a299746bb5164 server_cookie=176a4b5fa94c436c in_seq=0 out_seq=0 2024-04-26T21:46:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.418+0000 7fa6bf400640 10 client.4668 ms_handle_connect on v2:172.21.15.204:6834/4043819607 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6bf400640 1 -- 172.21.15.106:0/289040912 <== mds.0 v2:172.21.15.204:6834/4043819607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f3b582210 con 0x558f3b653120 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6bf400640 10 client.4668 handle_command_reply: tid=0 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 client.4668 shutdown 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 msgr2=0x558f3b637d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00dea0 tx=0x7fa6cc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 shutdown_connections 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b653120 0x558f3b6738b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b665db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b63dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 --2- 172.21.15.106:0/289040912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637d70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 >> 172.21.15.106:0/289040912 conn(0x558f3b5a4d10 msgr2=0x558f3b659780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6bf400640 0 client.4668 ms_handle_reset on v2:172.21.15.204:6834/4043819607 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 shutdown_connections 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 1 -- 172.21.15.106:0/289040912 wait complete. 2024-04-26T21:46:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.420+0000 7fa6e426e740 20 client.4668 trim_cache size 0 max 16384 2024-04-26T21:46:13.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.440+0000 7fa6e426e740 1 Processor -- start 2024-04-26T21:46:13.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.440+0000 7fa6e426e740 1 -- start start 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b63b270 0x558f3b60ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b64ce80 0x558f3b60f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc2005d0 con 0x558f3b6107b0 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1616e0 con 0x558f3b64ce80 2024-04-26T21:46:13.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc161d30 con 0x558f3b63b270 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35224/0 (socket says 172.21.15.106:35224) 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 -- 172.21.15.106:0/3567570860 learned_addr learned my addr 172.21.15.106:0/3567570860 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 -- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b63b270 msgr2=0x558f3b60ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b63b270 0x558f3b60ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 -- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b64ce80 msgr2=0x558f3b60f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b64ce80 0x558f3b60f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.441+0000 7fa6da800640 1 -- 172.21.15.106:0/3567570860 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3b63dee0 con 0x558f3b6107b0 2024-04-26T21:46:13.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6da800640 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc18fed6c9a4cf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6ca600640 1 -- 172.21.15.106:0/3567570860 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01a070 con 0x558f3b6107b0 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6ca600640 1 -- 172.21.15.106:0/3567570860 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc024070 con 0x558f3b6107b0 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6ca600640 1 -- 172.21.15.106:0/3567570860 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc014070 con 0x558f3b6107b0 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6e426e740 1 -- 172.21.15.106:0/3567570860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 msgr2=0x558f3b679250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.442+0000 7fa6e426e740 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc01e000 comp rx=0 tx=0).stop 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 -- 172.21.15.106:0/3567570860 shutdown_connections 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b64ce80 0x558f3b60f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b63b270 0x558f3b60ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 --2- 172.21.15.106:0/3567570860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b679250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 -- 172.21.15.106:0/3567570860 >> 172.21.15.106:0/3567570860 conn(0x558f3b5a4d10 msgr2=0x558f3b606e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 -- 172.21.15.106:0/3567570860 shutdown_connections 2024-04-26T21:46:13.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 -- 172.21.15.106:0/3567570860 wait complete. 2024-04-26T21:46:13.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 -- start start 2024-04-26T21:46:13.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.443+0000 7fa6e426e740 1 Processor -- start 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b6c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b64f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558f3b652fa0 con 0x558f3b6107b0 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3b652aa0 con 0x558f3b63b270 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558f3b6535f0 con 0x558f3b64ce80 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.444+0000 7fa6e426e740 10 client.0 _ll_register_callbacks cb 0x7fa6cc00b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:13.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35240/0 (socket says 172.21.15.106:35240) 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 -- 172.21.15.106:0/1744300547 learned_addr learned my addr 172.21.15.106:0/1744300547 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 -- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 msgr2=0x558f3b64f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b64f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 -- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 msgr2=0x558f3b6c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b6c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 -- 172.21.15.106:0/1744300547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3b64c2d0 con 0x558f3b6107b0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6da800640 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006dc0 tx=0x7fa6cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24636c40e522f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.445+0000 7fa6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01a040 con 0x558f3b6107b0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6cc041ab0 con 0x558f3b6107b0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc024070 con 0x558f3b6107b0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc041cd0 con 0x558f3b6107b0 2024-04-26T21:46:13.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 10 client.4674 fetch_fsmap learned FSMap version 7 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558f3b60fb60 con 0x558f3b6107b0 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc024070 con 0x558f3b6107b0 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fa6cc02f040 con 0x558f3b6107b0 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 10 client.4674 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 10 client.4674 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 20 client.4674 populate_metadata read hostname 'smithi106' 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b60f380 0x558f3b6d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 4 client.4674 mds_command: new command op to 34166 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.446+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 --> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] -- command(tid 0: {"prefix": "status"}) -- 0x558f3b57c080 con 0x558f3b60f380 2024-04-26T21:46:13.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.447+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b60f380 0x558f3b6d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:13.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.448+0000 7fa6d9e00640 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b60f380 0x558f3b6d3310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37324212ac90ffb server_cookie=e76b8ce3ee33256c in_seq=0 out_seq=0 2024-04-26T21:46:13.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.448+0000 7fa6bf400640 10 client.4674 ms_handle_connect on v2:172.21.15.204:6834/4043819607 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6bf400640 1 -- 172.21.15.106:0/1744300547 <== mds.0 v2:172.21.15.204:6834/4043819607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558f3b57c080 con 0x558f3b60f380 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6bf400640 10 client.4674 handle_command_reply: tid=0 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 client.4674 shutdown 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 msgr2=0x558f3b637170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006dc0 tx=0x7fa6cc015000 comp rx=0 tx=0).stop 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 shutdown_connections 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:6834/4043819607,v1:172.21.15.204:6835/4043819607] conn(0x558f3b60f380 0x558f3b6d3310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558f3b64ce80 0x558f3b64f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3b63b270 0x558f3b6c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 --2- 172.21.15.106:0/1744300547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3b6107b0 0x558f3b637170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6bf400640 0 client.4674 ms_handle_reset on v2:172.21.15.204:6834/4043819607 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 >> 172.21.15.106:0/1744300547 conn(0x558f3b5a4d10 msgr2=0x558f3b652170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 shutdown_connections 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 1 -- 172.21.15.106:0/1744300547 wait complete. 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.449+0000 7fa6e426e740 20 client.4674 trim_cache size 0 max 16384 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e6ee84ed-cbe5-4ff1-aa0d-97a291446523", 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 34166, 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 16.758612801000002, 2024-04-26T21:46:13.453 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 7, 2024-04-26T21:46:13.454 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-26T21:46:13.454 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-26T21:46:13.454 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 21.931378092999999 2024-04-26T21:46:13.454 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 msgr2=0x7fa6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 0x7fa6b806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006fd0 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 msgr2=0x7fa6dc14aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c400 tx=0x7fa6c400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 shutdown_connections 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6b806c8f0 0x7fa6b806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc0d0e70 0x7fa6dc151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc0b5d40 0x7fa6dc14aea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 --2- 172.21.15.106:0/2982014766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc0b5660 0x7fa6dc160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 >> 172.21.15.106:0/2982014766 conn(0x7fa6dc06c740 msgr2=0x7fa6dc06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 shutdown_connections 2024-04-26T21:46:13.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:13.450+0000 7fa6e0a00640 1 -- 172.21.15.106:0/2982014766 wait complete. 2024-04-26T21:46:13.467 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:46:13.477 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:46:13.477 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:46:13.477 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:46:13.477 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T21:46:13.533 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T21:46:13.534 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T21:46:13.534 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-26T21:46:13.534 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:46:13.534 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:13.534 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-26T21:46:13.534 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:46:13.534 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:13.534 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-26T21:46:13.534 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:46:13.535 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:13.535 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-26T21:46:13.535 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:46:13.535 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:13.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:13.535 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:46:13.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:13.593 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-04-26T21:46:13.681 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-04-26T21:46:13.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:46:13.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:13.685 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:46:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:13.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-26T21:46:13.835 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2024-04-26T21:46:13.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:46:13.840 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T21:46:13.840 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:14.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.042+0000 7fa90b200640 1 Processor -- start 2024-04-26T21:46:14.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.042+0000 7fa90b200640 1 -- start start 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90b200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa90c15f1e0 0x7fa90c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa90c0d5b60 0x7fa90c0d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90b200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90c15f5c0 con 0x7fa90c0d2fc0 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90b200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa90c0b8670 con 0x7fa90c15f1e0 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa90c0b8950 con 0x7fa90c0d5b60 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa909e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa90c15f1e0 0x7fa90c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90a800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa90c0d2fc0 0x7fa90c0cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36430/0 (socket says 172.21.15.106:36430) 2024-04-26T21:46:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa90a800640 1 -- 172.21.15.106:0/1340798158 learned_addr learned my addr 172.21.15.106:0/1340798158 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.044+0000 7fa903e00640 1 --2- 172.21.15.106:0/1340798158 >> v2:172.21.15.204:3300/0 conn(0x7fa90c0d5b60 0x7fa90c0d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.045+0000 7fa909e00640 1 -- 172.21.15.106:0/1340798158 >> v2:172.21.15.204:3300/0 conn(0x7fa90c0d5b60 msgr2=0x7fa90c0d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.045+0000 7fa909e00640 1 --2- 172.21.15.106:0/1340798158 >> v2:172.21.15.204:3300/0 conn(0x7fa90c0d5b60 0x7fa90c0d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.045+0000 7fa909e00640 1 -- 172.21.15.106:0/1340798158 >> v1:172.21.15.106:6789/0 conn(0x7fa90c0d2fc0 legacy=0x7fa90c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.045+0000 7fa909e00640 1 -- 172.21.15.106:0/1340798158 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c0b8c30 con 0x7fa90c15f1e0 2024-04-26T21:46:14.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.046+0000 7fa909e00640 1 --2- 172.21.15.106:0/1340798158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c0b7d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa90c08db50 tx=0x7fa8fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33f82cb704d0871 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:14.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.046+0000 7fa909400640 1 -- 172.21.15.106:0/1340798158 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc037070 con 0x7fa90c15f1e0 2024-04-26T21:46:14.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.046+0000 7fa909400640 1 -- 172.21.15.106:0/1340798158 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8fc032440 con 0x7fa90c15f1e0 2024-04-26T21:46:14.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.046+0000 7fa909400640 1 -- 172.21.15.106:0/1340798158 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc032810 con 0x7fa90c15f1e0 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 -- 172.21.15.106:0/1340798158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 msgr2=0x7fa90c0b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 --2- 172.21.15.106:0/1340798158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c0b7d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa90c08db50 tx=0x7fa8fc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 -- 172.21.15.106:0/1340798158 shutdown_connections 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 --2- 172.21.15.106:0/1340798158 >> v2:172.21.15.204:3300/0 conn(0x7fa90c0d5b60 0x7fa90c0d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 --2- 172.21.15.106:0/1340798158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c0b7d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 -- 172.21.15.106:0/1340798158 >> 172.21.15.106:0/1340798158 conn(0x7fa90c09e2d0 msgr2=0x7fa90c0d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 -- 172.21.15.106:0/1340798158 shutdown_connections 2024-04-26T21:46:14.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.047+0000 7fa90b200640 1 -- 172.21.15.106:0/1340798158 wait complete. 2024-04-26T21:46:14.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.049+0000 7fa90b200640 1 Processor -- start 2024-04-26T21:46:14.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.049+0000 7fa90b200640 1 -- start start 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa90c0d2fc0 0x7fa90c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa90c0d5b60 0x7fa90c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c0b82d0 con 0x7fa90c15f1e0 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c0b91f0 con 0x7fa90c0d2fc0 2024-04-26T21:46:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa90c0b8450 con 0x7fa90c0d5b60 2024-04-26T21:46:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35258/0 (socket says 172.21.15.106:35258) 2024-04-26T21:46:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa90a800640 1 -- 172.21.15.106:0/3519749531 learned_addr learned my addr 172.21.15.106:0/3519749531 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa903e00640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa90c0d5b60 0x7fa90c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.050+0000 7fa909e00640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa90c0d2fc0 0x7fa90c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 -- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa90c0d5b60 msgr2=0x7fa90c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa90c0d5b60 0x7fa90c154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 -- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa90c0d2fc0 msgr2=0x7fa90c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa90c0d2fc0 0x7fa90c1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 -- 172.21.15.106:0/3519749531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c1560c0 con 0x7fa90c15f1e0 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90a800640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa90400cbf0 tx=0x7fa90400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904017070 con 0x7fa90c15f1e0 2024-04-26T21:46:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90400ad40 con 0x7fa90c15f1e0 2024-04-26T21:46:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904012600 con 0x7fa90c15f1e0 2024-04-26T21:46:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c1563a0 con 0x7fa90c15f1e0 2024-04-26T21:46:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.051+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c156960 con 0x7fa90c15f1e0 2024-04-26T21:46:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.053+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90c0693c0 con 0x7fa90c15f1e0 2024-04-26T21:46:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.053+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa90401a070 con 0x7fa90c15f1e0 2024-04-26T21:46:14.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.054+0000 7fa902000640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 0x7fa8e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:14.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.054+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa90408b790 con 0x7fa90c15f1e0 2024-04-26T21:46:14.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.055+0000 7fa909e00640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 0x7fa8e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:14.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.055+0000 7fa909e00640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 0x7fa8e806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc034000 tx=0x7fa8fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.059+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa904059600 con 0x7fa90c15f1e0 2024-04-26T21:46:14.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.301+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa90c0c83c0 con 0x7fa90c15f1e0 2024-04-26T21:46:14.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.302+0000 7fa902000640 1 -- 172.21.15.106:0/3519749531 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fa90405d900 con 0x7fa90c15f1e0 2024-04-26T21:46:14.305 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:14.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 msgr2=0x7fa8e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 0x7fa8e806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc034000 tx=0x7fa8fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:14.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 msgr2=0x7fa90c15f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa90400cbf0 tx=0x7fa90400a430 comp rx=0 tx=0).stop 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 shutdown_connections 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa90c15f1e0 0x7fa90c15f7b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa8e806c8f0 0x7fa8e806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa90c0d5b60 0x7fa90c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 --2- 172.21.15.106:0/3519749531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa90c0d2fc0 0x7fa90c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 >> 172.21.15.106:0/3519749531 conn(0x7fa90c09e2d0 msgr2=0x7fa90c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.304+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 shutdown_connections 2024-04-26T21:46:14.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14.305+0000 7fa90b200640 1 -- 172.21.15.106:0/3519749531 wait complete. 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:46:14.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:14.326 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:14.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:14.326 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-26T21:46:14.374 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:14.375 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:46:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42584sec preferred_lft 42584sec 2024-04-26T21:46:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-26T21:46:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:14.377 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:46:14.377 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:14.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:14.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:14.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:14.868 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:14.944 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-26T21:46:14.947 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:14.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:14.948 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-26T21:46:14.996 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:15.053 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-26T21:46:15.053 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:15.053 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:46:15.053 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:46:15.054 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:46:15.055 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:15.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:15.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:15.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:15.839 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:46:15.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:15.891 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-26T21:46:15.940 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:15.940 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:15.940 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:15.940 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:46:15.940 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:16.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:16.113 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:46:16.114 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:16.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:16.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:16.399 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:16.399 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:16.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-26T21:46:16.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:46:16.399 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:16.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:16.598 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T21:46:16.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:16.598 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:16.641 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:16.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:16.641 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:16.736 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:46:16.814 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:46:16.859 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:16.860 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:16.861 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:16.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:16.862 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:46:16.959 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:46:16.959 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T21:46:16.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:46:17.031 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:46:17.073 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:46:17.074 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:17.075 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:17.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:17.076 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:46:17.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-26T21:46:17.153+0000 7f286871e480 -1 init, newargv = 0x563cb41f83b0 newargc=15 2024-04-26T21:46:17.155 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[41575]: starting ceph client 2024-04-26T21:46:17.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[41575]: starting fuse 2024-04-26T21:46:18.264 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:46:18.349 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:46:18.405 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:18.406 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:46:18.407 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:18.408 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:18.408 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:18.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.409 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:46:18.460 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-26T21:46:18.461 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:46:18.461 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-26T21:46:18.462 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-26T21:46:18.463 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:46:18.463 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:18.463 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-26T21:46:18.463 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-26T21:46:18.625 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.41575.asok 2024-04-26T21:46:18.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:18.642 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41575.asok 2024-04-26T21:46:18.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.643 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41575.asok status 2024-04-26T21:46:18.870 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:46:18.870 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "41575", 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4689, 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-26T21:46:18.871 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4689 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 2891939916 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:46:18.872 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 2891939916 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4689 192.168.0.1:0/2891939916", 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/2891939916", 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 7, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 20, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-26T21:46:18.873 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:46:18.885 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.073+0000 7fbb2ea00640 1 Processor -- start 2024-04-26T21:46:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.073+0000 7fbb2ea00640 1 -- start start 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb280d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb280c9520 0x7fbb280c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2815f800 con 0x7fbb280d3680 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ea00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbb280d2c70 con 0x7fbb280c9520 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb280d3480 con 0x7fbb2815f420 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb27400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbb280d3680 0x7fbb280d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51656/0 (socket says 172.21.15.106:51656) 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb27400640 1 -- 172.21.15.106:0/2471322101 learned_addr learned my addr 172.21.15.106:0/2471322101 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb2ca00640 1 --2- 172.21.15.106:0/2471322101 >> v2:172.21.15.106:3300/0 conn(0x7fbb280c9520 0x7fbb280c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.075+0000 7fbb27e00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb280d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.076+0000 7fbb2ca00640 1 -- 172.21.15.106:0/2471322101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 msgr2=0x7fbb280d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.076+0000 7fbb2ca00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb280d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.076+0000 7fbb2ca00640 1 -- 172.21.15.106:0/2471322101 >> v1:172.21.15.106:6789/0 conn(0x7fbb280d3680 legacy=0x7fbb280d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.076+0000 7fbb2ca00640 1 -- 172.21.15.106:0/2471322101 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb280d2ec0 con 0x7fbb280c9520 2024-04-26T21:46:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.076+0000 7fbb26a00640 1 -- 172.21.15.106:0/2471322101 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982625962 0 0) 0x7fbb2815f800 con 0x7fbb280d3680 2024-04-26T21:46:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.077+0000 7fbb2ca00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280c9520 0x7fbb280c99b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009d90 tx=0x7fbb2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5018e48a6426484b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.077+0000 7fbb26a00640 1 -- 172.21.15.106:0/2471322101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031600 con 0x7fbb280c9520 2024-04-26T21:46:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb26a00640 1 -- 172.21.15.106:0/2471322101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb200317a0 con 0x7fbb280c9520 2024-04-26T21:46:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb26a00640 1 -- 172.21.15.106:0/2471322101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031b70 con 0x7fbb280c9520 2024-04-26T21:46:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2471322101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280c9520 msgr2=0x7fbb280c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280c9520 0x7fbb280c99b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009d90 tx=0x7fbb2002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2471322101 shutdown_connections 2024-04-26T21:46:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280c9520 0x7fbb280c99b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2471322101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb280d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2471322101 >> 172.21.15.106:0/2471322101 conn(0x7fbb2809e3e0 msgr2=0x7fbb280d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2471322101 shutdown_connections 2024-04-26T21:46:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.078+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2471322101 wait complete. 2024-04-26T21:46:19.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.080+0000 7fbb2ea00640 1 Processor -- start 2024-04-26T21:46:19.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.080+0000 7fbb2ea00640 1 -- start start 2024-04-26T21:46:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 0x7fbb280b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb28160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb280c9000 con 0x7fbb280d3680 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb280c8d20 con 0x7fbb280c9520 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb2ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb280d0f30 con 0x7fbb2815f420 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb27400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.081+0000 7fbb27400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42010/0 (socket says 172.21.15.106:42010) 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 -- 172.21.15.106:0/2232806652 learned_addr learned my addr 172.21.15.106:0/2232806652 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb2ca00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb28160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27e00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 0x7fbb280b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 -- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 msgr2=0x7fbb28160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb28160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 -- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 msgr2=0x7fbb280b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 0x7fbb280b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 -- 172.21.15.106:0/2232806652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb280cce30 con 0x7fbb280d3680 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27e00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 0x7fbb280b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb27400640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c500 tx=0x7fbb1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10018070 con 0x7fbb280d3680 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10007580 con 0x7fbb280d3680 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.082+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004a20 con 0x7fbb280d3680 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.083+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb280cd110 con 0x7fbb280d3680 2024-04-26T21:46:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.083+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb280cde40 con 0x7fbb280d3680 2024-04-26T21:46:19.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.084+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb28069540 con 0x7fbb280d3680 2024-04-26T21:46:19.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.085+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb10004bc0 con 0x7fbb280d3680 2024-04-26T21:46:19.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.086+0000 7fbb1fe00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 0x7fbb0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.086+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb1008ae70 con 0x7fbb280d3680 2024-04-26T21:46:19.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.086+0000 7fbb27e00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 0x7fbb0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.087+0000 7fbb27e00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 0x7fbb0406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006fd0 tx=0x7fbb18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.091+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb10059290 con 0x7fbb280d3680 2024-04-26T21:46:19.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.339+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb280b98e0 con 0x7fbb280d3680 2024-04-26T21:46:19.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.340+0000 7fbb1fe00640 1 -- 172.21.15.106:0/2232806652 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fbb10058b90 con 0x7fbb280d3680 2024-04-26T21:46:19.341 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.343+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 msgr2=0x7fbb0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.343+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 0x7fbb0406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006fd0 tx=0x7fbb18008040 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.343+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 msgr2=0x7fbb281f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.343+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c500 tx=0x7fbb1000c9c0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 shutdown_connections 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb0406c8f0 0x7fbb0406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb2815f420 0x7fbb28160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb280d3680 0x7fbb281f5070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 --2- 172.21.15.106:0/2232806652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb280c9520 0x7fbb280b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 >> 172.21.15.106:0/2232806652 conn(0x7fbb2809e3e0 msgr2=0x7fbb280c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 shutdown_connections 2024-04-26T21:46:19.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19.344+0000 7fbb2ea00640 1 -- 172.21.15.106:0/2232806652 wait complete. 2024-04-26T21:46:19.363 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:46:19.363 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:19.363 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-26T21:46:19.363 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T21:46:19.364 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:19.364 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:46:19.364 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:46:19.364 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:19.364 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:46:19.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:19.364 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.412 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c49c:8bff:fe17:2a52/64 scope link 2024-04-26T21:46:19.413 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.413 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:19.413 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:19.413 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::480c:21ff:fe55:3af6/64 scope link 2024-04-26T21:46:19.413 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:19.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:19.414 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:19.539 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-26T21:46:19.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:19.544 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-26T21:46:19.594 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:46:19.595 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:19.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:19.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:19.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:19.803 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:46:19.851 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:46:19.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:19.852 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-26T21:46:19.899 INFO:teuthology.orchestra.run.smithi106.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T21:46:19.900 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:19.901 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:19.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:19.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T21:46:19.901 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:20.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:20.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:20.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:20.086 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether d2:ef:4b:fd:ce:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.164 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::d0ef:4bff:fefd:ce50/64 scope link 2024-04-26T21:46:20.165 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:20.168 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether d2:ef:4b:fd:ce:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::d0ef:4bff:fefd:ce50/64 scope link 2024-04-26T21:46:20.251 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:20.256 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T21:46:20.256 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:20.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:20.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:20.529 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:20.530 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:46:20.530 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.1 up 2024-04-26T21:46:20.530 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:46:20.530 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:46:20.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:20.731 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-26T21:46:20.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:20.731 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:20.772 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:46:20.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:20.773 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:20.866 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:46:20.949 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:46:21.003 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:21.004 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:21.005 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:21.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:21.006 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:46:21.055 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-26T21:46:21.055 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-26T21:46:21.055 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-26T21:46:21.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:46:21.137 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:21.179 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:46:21.179 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:46:21.179 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:46:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:46:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:21.182 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:21.182 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:21.182 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:21.182 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:21.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:21.183 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:46:21.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:2024-04-26T21:46:21.233+0000 7faea5eb2480 -1 init, newargv = 0x565440e875f0 newargc=15 2024-04-26T21:46:21.234 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:ceph-fuse[42141]: starting ceph client 2024-04-26T21:46:21.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi106.stderr:ceph-fuse[42141]: starting fuse 2024-04-26T21:46:21.263 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-26T21:46:21.263 INFO:teuthology.orchestra.run.smithi106.stdout:43 2024-04-26T21:46:21.263 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-26T21:46:21.264 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.2")) 2024-04-26T21:46:21.265 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-26T21:46:21.452 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.2.42141.asok 2024-04-26T21:46:21.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:21.469 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.42141.asok 2024-04-26T21:46:21.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:21.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.42141.asok status 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "2", 2024-04-26T21:46:21.695 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "42141", 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4707, 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4707 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:46:21.696 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.2:0", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 869405539 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.2:0", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 869405539 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4707 192.168.0.2:0/869405539", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.2:0/869405539", 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 7, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 20, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:46:21.697 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-26T21:46:21.698 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-26T21:46:21.698 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:46:21.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:21.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.909+0000 7f2f62800640 1 Processor -- start 2024-04-26T21:46:21.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.909+0000 7f2f62800640 1 -- start start 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.910+0000 7f2f62800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f5c145750 0x7f2f5c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f62800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5c073050 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f62800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5c073330 con 0x7f2f5c0cf730 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f62800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5c070220 con 0x7f2f5c145750 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f60a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f5c06fdc0 0x7f2f5c0cbd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51666/0 (socket says 172.21.15.106:51666) 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f60a00640 1 -- 172.21.15.106:0/1004484718 learned_addr learned my addr 172.21.15.106:0/1004484718 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f61400640 1 --2- 172.21.15.106:0/1004484718 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c145750 0x7f2f5c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936217961 0 0) 0x7f2f5c073050 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.911+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f44003520 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3046743455 0 0) 0x7f2f44003520 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c145750 msgr2=0x7f2f5c06f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 --2- 172.21.15.106:0/1004484718 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c145750 0x7f2f5c06f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 >> v1:172.21.15.204:6789/0 conn(0x7f2f5c0cf730 legacy=0x7f2f5c165870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c073610 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2f48001670 con 0x7f2f5c06fdc0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 >> v1:172.21.15.106:6789/0 conn(0x7f2f5c06fdc0 legacy=0x7f2f5c0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f44003a00 0x7f2f44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 0x7f2f44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f440051c0 0x7f2f4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c070220 con 0x7f2f44004800 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4400c0c0 con 0x7f2f440051c0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4400c2c0 con 0x7f2f44003a00 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.912+0000 7f2f60a00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 0x7f2f44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f61400640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f44003a00 0x7f2f44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f61e00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f440051c0 0x7f2f4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f60a00640 1 -- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f44003a00 msgr2=0x7f2f44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f60a00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f44003a00 0x7f2f44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f60a00640 1 -- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f440051c0 msgr2=0x7f2f4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f60a00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f440051c0 0x7f2f4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f60a00640 1 -- 172.21.15.106:0/1004484718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f44003520 con 0x7f2f44004800 2024-04-26T21:46:21.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.913+0000 7f2f61e00640 1 -- 172.21.15.106:0/1004484718 reap_dead start 2024-04-26T21:46:21.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.914+0000 7f2f60a00640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 0x7f2f44004c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c110 tx=0x7f2f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fb782f53316575 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.914+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48007c40 con 0x7f2f44004800 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.914+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f48005af0 con 0x7f2f44004800 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.914+0000 7f2f5be00640 1 -- 172.21.15.106:0/1004484718 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48007070 con 0x7f2f44004800 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 -- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 msgr2=0x7f2f44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 0x7f2f44004c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c110 tx=0x7f2f44005830 comp rx=0 tx=0).stop 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 -- 172.21.15.106:0/1004484718 shutdown_connections 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 --2- 172.21.15.106:0/1004484718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f44004800 0x7f2f44004c80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 -- 172.21.15.106:0/1004484718 >> 172.21.15.106:0/1004484718 conn(0x7f2f5c0d0ae0 msgr2=0x7f2f5c0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 -- 172.21.15.106:0/1004484718 shutdown_connections 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.915+0000 7f2f62800640 1 -- 172.21.15.106:0/1004484718 wait complete. 2024-04-26T21:46:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.917+0000 7f2f62800640 1 Processor -- start 2024-04-26T21:46:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.917+0000 7f2f62800640 1 -- start start 2024-04-26T21:46:21.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 0x7f2f5c144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c0b0c30 0x7f2f5c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c073eb0 con 0x7f2f5c15ded0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c073bd0 con 0x7f2f44004800 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f62800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5c0cbd60 con 0x7f2f5c0b0c30 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f60a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f60a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42024/0 (socket says 172.21.15.106:42024) 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f60a00640 1 -- 172.21.15.106:0/919818518 learned_addr learned my addr 172.21.15.106:0/919818518 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f61400640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 0x7f2f5c144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.918+0000 7f2f61e00640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c0b0c30 0x7f2f5c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 -- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c0b0c30 msgr2=0x7f2f5c0b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c0b0c30 0x7f2f5c0b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 -- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 msgr2=0x7f2f5c144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 0x7f2f5c144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 -- 172.21.15.106:0/919818518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c15e390 con 0x7f2f5c15ded0 2024-04-26T21:46:21.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f61400640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 0x7f2f5c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:21.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f60a00640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c230 tx=0x7f2f5c073220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48037040 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f48041070 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004a00 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c15e670 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.919+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c15ec30 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.921+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5c0693c0 con 0x7f2f5c15ded0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.921+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f48004ba0 con 0x7f2f5c15ded0 2024-04-26T21:46:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.922+0000 7f2f59600640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.922+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f480ab9b0 con 0x7f2f5c15ded0 2024-04-26T21:46:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.923+0000 7f2f61400640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.923+0000 7f2f61400640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c06aad0 tx=0x7f2f50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:21.927+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f48079dd0 con 0x7f2f5c15ded0 2024-04-26T21:46:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.170+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f5c13d9c0 con 0x7f2f5c15ded0 2024-04-26T21:46:22.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.171+0000 7f2f59600640 1 -- 172.21.15.106:0/919818518 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2f48005ab0 con 0x7f2f5c15ded0 2024-04-26T21:46:22.172 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.173+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 msgr2=0x7f2f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c06aad0 tx=0x7f2f50008040 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 msgr2=0x7f2f5c13b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c230 tx=0x7f2f5c073220 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 shutdown_connections 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c0b0c30 0x7f2f5c0b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f5c15ded0 0x7f2f5c13b490 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 --2- 172.21.15.106:0/919818518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f44004800 0x7f2f5c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 >> 172.21.15.106:0/919818518 conn(0x7f2f5c0d0ae0 msgr2=0x7f2f5c0cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 shutdown_connections 2024-04-26T21:46:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:22.174+0000 7f2f62800640 1 -- 172.21.15.106:0/919818518 wait complete. 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:22.193 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:46:22.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:22.194 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42597sec preferred_lft 42597sec 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:22.254 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr flush dev ceph-brx 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set ceph-brx up 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:46:22.254 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:22.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:22.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:22.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:22.773 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:22.837 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-26T21:46:22.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:22.841 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-04-26T21:46:22.899 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-04-26T21:46:22.899 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:22.899 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:46:22.899 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:46:22.899 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:46:22.900 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:23.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:23.655 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:23.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:23.665 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:46:23.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:23.717 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-04-26T21:46:23.768 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:23.768 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:23.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:23.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T21:46:23.768 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:23.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:23.953 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T21:46:23.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T21:46:23.954 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T21:46:23.954 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:24.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:24.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:24.238 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:24.239 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:24.239 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-04-26T21:46:24.239 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:46:24.239 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:24.332 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:24.428 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:24.438 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T21:46:24.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:24.438 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:24.482 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:46:24.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:24.483 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:24.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:46:24.638 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:24.684 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:46:24.684 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:46:24.684 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:46:24.685 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:46:24.686 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:46:24.687 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:46:24.687 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:46:24.687 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:24.687 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:24.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:24.688 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:46:24.783 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:46:24.783 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T21:46:24.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:46:24.863 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:24.903 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:46:24.903 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:46:24.903 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:46:24.904 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:46:24.905 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:46:24.906 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:46:24.906 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:24.906 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:24.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:24.907 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:46:24.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-04-26T21:46:24.973+0000 7f70e83e1480 -1 init, newargv = 0x563d1693e340 newargc=15 2024-04-26T21:46:24.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[39289]: starting ceph client 2024-04-26T21:46:24.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[39289]: starting fuse 2024-04-26T21:46:24.991 INFO:teuthology.orchestra.run.smithi204.stdout:41 2024-04-26T21:46:24.992 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T21:46:24.992 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-04-26T21:46:24.993 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-04-26T21:46:25.181 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.39289.asok 2024-04-26T21:46:25.185 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:25.197 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39289.asok 2024-04-26T21:46:25.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:25.198 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39289.asok status 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T21:46:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "39289", 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 34394, 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 34394 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 3679904521 2024-04-26T21:46:25.418 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 3679904521 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.34394 192.168.0.1:0/3679904521", 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/3679904521", 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 7, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 20, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-04-26T21:46:25.419 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:46:25.431 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.642+0000 7f0f0fe00640 1 Processor -- start 2024-04-26T21:46:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.642+0000 7f0f0fe00640 1 -- start start 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.643+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f1006fa90 0x7f0f1015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.643+0000 7f0f0fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f1006ed40 0x7f0f100cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.643+0000 7f0f0fe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f1006fef0 con 0x7f0f10073720 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.643+0000 7f0f0fe00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100701a0 con 0x7f0f1006ed40 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.643+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1006df80 con 0x7f0f1006fa90 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0ea00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0f10073720 0x7f0f10073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51672/0 (socket says 172.21.15.106:51672) 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f1006ed40 0x7f0f100cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0ea00640 1 -- 172.21.15.106:0/3909470722 learned_addr learned my addr 172.21.15.106:0/3909470722 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0f400640 1 -- 172.21.15.106:0/3909470722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f1006fa90 msgr2=0x7f0f1015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f06000640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f1006fa90 0x7f0f1015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0f400640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f1006fa90 0x7f0f1015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0f400640 1 -- 172.21.15.106:0/3909470722 >> v1:172.21.15.106:6789/0 conn(0x7f0f10073720 legacy=0x7f0f10073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0e000640 1 -- 172.21.15.106:0/3909470722 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841548969 0 0) 0x7f0f1006fef0 con 0x7f0f10073720 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.644+0000 7f0f0f400640 1 -- 172.21.15.106:0/3909470722 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1006e260 con 0x7f0f1006ed40 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.645+0000 7f0f0f400640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006ed40 0x7f0f100cb130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebb10ba3ba2b6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.646+0000 7f0f0e000640 1 -- 172.21.15.106:0/3909470722 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007740 con 0x7f0f1006ed40 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.646+0000 7f0f0e000640 1 -- 172.21.15.106:0/3909470722 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f080078e0 con 0x7f0f1006ed40 2024-04-26T21:46:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.646+0000 7f0f0e000640 1 -- 172.21.15.106:0/3909470722 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007cb0 con 0x7f0f1006ed40 2024-04-26T21:46:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.646+0000 7f0f0fe00640 1 -- 172.21.15.106:0/3909470722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006ed40 msgr2=0x7f0f100cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.646+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006ed40 0x7f0f100cb130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0802a660 comp rx=0 tx=0).stop 2024-04-26T21:46:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 -- 172.21.15.106:0/3909470722 shutdown_connections 2024-04-26T21:46:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006ed40 0x7f0f100cb130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/3909470722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f1006fa90 0x7f0f1015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 -- 172.21.15.106:0/3909470722 >> 172.21.15.106:0/3909470722 conn(0x7f0f100d0ae0 msgr2=0x7f0f100d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:25.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 -- 172.21.15.106:0/3909470722 shutdown_connections 2024-04-26T21:46:25.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.647+0000 7f0f0fe00640 1 -- 172.21.15.106:0/3909470722 wait complete. 2024-04-26T21:46:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.649+0000 7f0f0fe00640 1 Processor -- start 2024-04-26T21:46:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.649+0000 7f0f0fe00640 1 -- start start 2024-04-26T21:46:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1006ed40 0x7f0f100afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f10073720 0x7f0f100aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1006f1c0 con 0x7f0f1006fa90 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10146910 con 0x7f0f1006ed40 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f0fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100c7130 con 0x7f0f10073720 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f06000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f06000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42048/0 (socket says 172.21.15.106:42048) 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.650+0000 7f0f06000640 1 -- 172.21.15.106:0/2377468822 learned_addr learned my addr 172.21.15.106:0/2377468822 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f0f400640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f10073720 0x7f0f100aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:25.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 -- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f10073720 msgr2=0x7f0f100aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f10073720 0x7f0f100aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 -- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1006ed40 msgr2=0x7f0f100afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1006ed40 0x7f0f100afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 -- 172.21.15.106:0/2377468822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100c7410 con 0x7f0f1006fa90 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f06000640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c7f0 tx=0x7f0ef800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8015070 con 0x7f0f1006fa90 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef8004220 con 0x7f0f1006fa90 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.651+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f10135410 con 0x7f0f1006fa90 2024-04-26T21:46:25.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.652+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8007620 con 0x7f0f1006fa90 2024-04-26T21:46:25.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.652+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f100c76f0 con 0x7f0f1006fa90 2024-04-26T21:46:25.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.653+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100693c0 con 0x7f0f1006fa90 2024-04-26T21:46:25.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.654+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef80079d0 con 0x7f0f1006fa90 2024-04-26T21:46:25.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.655+0000 7f0f07400640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 0x7f0eec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:25.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.655+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ef808bc90 con 0x7f0f1006fa90 2024-04-26T21:46:25.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.655+0000 7f0f0ea00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 0x7f0eec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:25.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.656+0000 7f0f0ea00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 0x7f0eec06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f00007fd0 tx=0x7f0f00007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.660+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ef805a030 con 0x7f0f1006fa90 2024-04-26T21:46:25.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.903+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0f10144720 con 0x7f0f1006fa90 2024-04-26T21:46:25.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.904+0000 7f0f07400640 1 -- 172.21.15.106:0/2377468822 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f0ef8059930 con 0x7f0f1006fa90 2024-04-26T21:46:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 msgr2=0x7f0eec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 0x7f0eec06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f00007fd0 tx=0x7f0f00007960 comp rx=0 tx=0).stop 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 msgr2=0x7f0f100c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c7f0 tx=0x7f0ef800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 shutdown_connections 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0eec06c8f0 0x7f0eec06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f10073720 0x7f0f100aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1006fa90 0x7f0f100c6660 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/2377468822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1006ed40 0x7f0f100afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 >> 172.21.15.106:0/2377468822 conn(0x7f0f100d0ae0 msgr2=0x7f0f100b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 shutdown_connections 2024-04-26T21:46:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:25.907+0000 7f0f0fe00640 1 -- 172.21.15.106:0/2377468822 wait complete. 2024-04-26T21:46:25.926 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:46:25.926 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:25.926 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:25.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:46:25.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:25.927 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-04-26T21:46:25.983 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:25.983 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:25.983 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:25.983 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.983 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42593sec preferred_lft 42593sec 2024-04-26T21:46:25.984 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ce:52:1d:14:cf:6d brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::2818:afff:fe5c:ca4c/64 scope link 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ce:52:1d:14:cf:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::cc52:1dff:fe14:cf6d/64 scope link tentative 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:25.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:25.986 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:26.059 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-26T21:46:26.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:26.063 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-04-26T21:46:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-04-26T21:46:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:46:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:46:26.113 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:46:26.114 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:26.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:26.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:26.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:26.326 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:46:26.374 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:46:26.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:26.375 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-04-26T21:46:26.469 INFO:teuthology.orchestra.run.smithi204.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T21:46:26.471 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:26.471 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:26.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:46:26.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T21:46:26.471 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:26.580 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:26.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:26.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:26.659 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:26.741 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether f6:22:e3:2f:e5:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:46:26.742 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:46:26.742 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.742 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f422:e3ff:fe2f:e5c1/64 scope link 2024-04-26T21:46:26.742 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:26.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether f6:22:e3:2f:e5:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:46:26.826 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:46:26.827 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.827 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f422:e3ff:fe2f:e5c1/64 scope link 2024-04-26T21:46:26.827 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:26.830 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T21:46:26.830 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:26.830 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:26.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T21:46:26.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:46:26.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T21:46:26.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T21:46:26.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T21:46:26.831 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:26.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:27.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:27.095 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:27.095 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:46:27.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.1 up 2024-04-26T21:46:27.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:46:27.096 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:46:27.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:27.288 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:27.298 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-26T21:46:27.298 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:27.298 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:27.343 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:46:27.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:27.344 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:27.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:46:27.526 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:27.580 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:46:27.580 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:46:27.580 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:46:27.580 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:46:27.580 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:46:27.581 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:46:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:46:27.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:27.583 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:46:27.631 INFO:teuthology.orchestra.run.smithi204.stdout:41 2024-04-26T21:46:27.632 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-26T21:46:27.632 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-26T21:46:27.639 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:46:27.707 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:46:27.748 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:46:27.749 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:46:27.750 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:46:27.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:27.751 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:46:27.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:2024-04-26T21:46:27.808+0000 7f1a18dee480 -1 init, newargv = 0x55ff3e00c890 newargc=15 2024-04-26T21:46:27.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:ceph-fuse[39745]: starting ceph client 2024-04-26T21:46:27.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:ceph-fuse[39745]: starting fuse 2024-04-26T21:46:27.835 INFO:teuthology.orchestra.run.smithi204.stdout:41 2024-04-26T21:46:27.835 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-04-26T21:46:27.835 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-26T21:46:27.835 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:46:27.835 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-04-26T21:46:27.835 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-04-26T21:46:27.835 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-04-26T21:46:27.835 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.3")) 2024-04-26T21:46:27.837 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-04-26T21:46:28.019 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.3.39745.asok 2024-04-26T21:46:28.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:28.035 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39745.asok 2024-04-26T21:46:28.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.035 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39745.asok status 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "3", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "39745", 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-04-26T21:46:28.256 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 34403, 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 34403 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.2:0", 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 2768549274 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.2:0", 2024-04-26T21:46:28.257 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 2768549274 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.34403 192.168.0.2:0/2768549274", 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.2:0/2768549274", 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 7, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 20, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-04-26T21:46:28.258 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:46:28.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.270 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-26T21:46:28.270 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:28.330 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-26T21:46:28.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:28.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:28.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.421 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-26T21:46:28.421 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:28.475 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-26T21:46:28.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:28.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:28.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.571 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-04-26T21:46:28.571 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:28.628 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-04-26T21:46:28.628 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:28.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.628 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:28.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.713 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-04-26T21:46:28.714 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:28.763 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-04-26T21:46:28.763 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:28.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:28.763 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:28.842 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T21:46:28.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:29.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.059+0000 7fa9cf200640 1 Processor -- start 2024-04-26T21:46:29.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.059+0000 7fa9cf200640 1 -- start start 2024-04-26T21:46:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa9d00d0e70 0x7fa9d00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 0x7fa9d00b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cf200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d0146a60 con 0x7fa9d0146680 2024-04-26T21:46:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d006e970 con 0x7fa9d00d0e70 2024-04-26T21:46:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d00b60a0 con 0x7fa9d00b3b20 2024-04-26T21:46:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cde00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa9d0146680 0x7fa9d00b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43564/0 (socket says 172.21.15.106:43564) 2024-04-26T21:46:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.060+0000 7fa9cde00640 1 -- 172.21.15.106:0/2218520177 learned_addr learned my addr 172.21.15.106:0/2218520177 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341480171 0 0) 0x7fa9d0146a60 con 0x7fa9d0146680 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9b4003520 con 0x7fa9d0146680 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9ce800640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 0x7fa9d00b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/2218520177 >> v2:172.21.15.204:3300/0 conn(0x7fa9d00d0e70 0x7fa9d00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3991599455 0 0) 0x7fa9b4003520 con 0x7fa9d0146680 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 msgr2=0x7fa9d00b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 0x7fa9d00b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 >> v2:172.21.15.204:3300/0 conn(0x7fa9d00d0e70 msgr2=0x7fa9d00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 --2- 172.21.15.106:0/2218520177 >> v2:172.21.15.204:3300/0 conn(0x7fa9d00d0e70 0x7fa9d00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d006dd60 con 0x7fa9d0146680 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fa9c0001670 con 0x7fa9d0146680 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 >> v1:172.21.15.106:6789/0 conn(0x7fa9d0146680 legacy=0x7fa9d00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b40039b0 0x7fa9b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9b4004800 0x7fa9b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 0x7fa9b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d00b60a0 con 0x7fa9b4004800 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b400b9f0 con 0x7fa9b40039b0 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.061+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b400bbf0 con 0x7fa9d00b3b20 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9b4004800 0x7fa9b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9cde00640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b40039b0 0x7fa9b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 -- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 msgr2=0x7fa9b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00b3b20 0x7fa9b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 -- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b40039b0 msgr2=0x7fa9b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b40039b0 0x7fa9b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.062+0000 7fa9c7e00640 1 -- 172.21.15.106:0/2218520177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b4003520 con 0x7fa9b4004800 2024-04-26T21:46:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.063+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9b4004800 0x7fa9b4004c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800ba00 tx=0x7fa9b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32afe6e2aa390f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.063+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8030070 con 0x7fa9b4004800 2024-04-26T21:46:29.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.063+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b802adb0 con 0x7fa9b4004800 2024-04-26T21:46:29.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.063+0000 7fa9cd400640 1 -- 172.21.15.106:0/2218520177 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800b600 con 0x7fa9b4004800 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 -- 172.21.15.106:0/2218520177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9b4004800 msgr2=0x7fa9b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 --2- 172.21.15.106:0/2218520177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9b4004800 0x7fa9b4004c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800ba00 tx=0x7fa9b4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9ce800640 1 -- 172.21.15.106:0/2218520177 reap_dead start 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 -- 172.21.15.106:0/2218520177 shutdown_connections 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 -- 172.21.15.106:0/2218520177 >> 172.21.15.106:0/2218520177 conn(0x7fa9d006c740 msgr2=0x7fa9d006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 -- 172.21.15.106:0/2218520177 shutdown_connections 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.064+0000 7fa9cf200640 1 -- 172.21.15.106:0/2218520177 wait complete. 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.066+0000 7fa9cf200640 1 Processor -- start 2024-04-26T21:46:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.066+0000 7fa9cf200640 1 -- start start 2024-04-26T21:46:29.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 0x7fa9d015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00cc620 0x7fa9d00ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d006e320 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d00d17f0 con 0x7fa9b4004800 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9cf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d006e600 con 0x7fa9d00cc620 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46736/0 (socket says 172.21.15.106:46736) 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.067+0000 7fa9c7e00640 1 -- 172.21.15.106:0/3397778530 learned_addr learned my addr 172.21.15.106:0/3397778530 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9ce800640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00cc620 0x7fa9d00ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9cde00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 0x7fa9d015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 -- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00cc620 msgr2=0x7fa9d00ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00cc620 0x7fa9d00ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 -- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 msgr2=0x7fa9d015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 0x7fa9d015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 -- 172.21.15.106:0/3397778530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d00caf90 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9cde00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 0x7fa9d015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c7e00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a8a0 tx=0x7fa9b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8043020 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b803f680 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.068+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800b600 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.069+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d00cb270 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.069+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d00cb830 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.070+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d00693c0 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.071+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9b8046020 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.072+0000 7fa9c6000640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 0x7fa9ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.072+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9b80ac860 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.072+0000 7fa9cde00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 0x7fa9ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.072+0000 7fa9cde00640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 0x7fa9ac06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0005990 tx=0x7fa9c0001db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.076+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9b804d030 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.278+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9d00b4e90 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.280+0000 7fa9c6000640 1 -- 172.21.15.106:0/3397778530 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7fa9b807ab60 con 0x7fa9d00cc1a0 2024-04-26T21:46:29.282 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:29.282 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:29.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.283+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 msgr2=0x7fa9ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.283+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 0x7fa9ac06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0005990 tx=0x7fa9c0001db0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.283+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 msgr2=0x7fa9d00ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.283+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a8a0 tx=0x7fa9b8004730 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 shutdown_connections 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa9ac06c8f0 0x7fa9ac06edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9d00cc620 0x7fa9d00ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9d00cc1a0 0x7fa9d00ca330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 --2- 172.21.15.106:0/3397778530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4004800 0x7fa9d015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 >> 172.21.15.106:0/3397778530 conn(0x7fa9d006c740 msgr2=0x7fa9d00cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 shutdown_connections 2024-04-26T21:46:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.284+0000 7fa9cf200640 1 -- 172.21.15.106:0/3397778530 wait complete. 2024-04-26T21:46:29.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:29.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.507+0000 7f7575400640 1 Processor -- start 2024-04-26T21:46:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.508+0000 7f7575400640 1 -- start start 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.509+0000 7f7575400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f75700d19c0 0x7f75700d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.509+0000 7f7575400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75700b6170 0x7f75700c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.509+0000 7f7575400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75700d3830 con 0x7f75700d1da0 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.509+0000 7f7575400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f75700d40d0 con 0x7f75700b6170 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.509+0000 7f7575400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f75700d3b10 con 0x7f75700d19c0 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75700b6170 0x7f75700c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75700b6170 0x7f75700c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46752/0 (socket says 172.21.15.106:46752) 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 -- 172.21.15.106:0/3277518210 learned_addr learned my addr 172.21.15.106:0/3277518210 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756e800640 1 --2- 172.21.15.106:0/3277518210 >> v2:172.21.15.204:3300/0 conn(0x7f75700d19c0 0x7f75700d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 -- 172.21.15.106:0/3277518210 >> v2:172.21.15.204:3300/0 conn(0x7f75700d19c0 msgr2=0x7f75700d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 --2- 172.21.15.106:0/3277518210 >> v2:172.21.15.204:3300/0 conn(0x7f75700d19c0 0x7f75700d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 -- 172.21.15.106:0/3277518210 >> v1:172.21.15.204:6789/0 conn(0x7f75700d1da0 legacy=0x7f7570162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.510+0000 7f756de00640 1 -- 172.21.15.106:0/3277518210 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75700d3df0 con 0x7f75700b6170 2024-04-26T21:46:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.511+0000 7f756de00640 1 --2- 172.21.15.106:0/3277518210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700b6170 0x7f75700c8c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f757006ca80 tx=0x7f755802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470606879d16891e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.511+0000 7f756d400640 1 -- 172.21.15.106:0/3277518210 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558036070 con 0x7f75700b6170 2024-04-26T21:46:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f756d400640 1 -- 172.21.15.106:0/3277518210 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7558031600 con 0x7f75700b6170 2024-04-26T21:46:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f756d400640 1 -- 172.21.15.106:0/3277518210 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580319d0 con 0x7f75700b6170 2024-04-26T21:46:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 -- 172.21.15.106:0/3277518210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700b6170 msgr2=0x7f75700c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 --2- 172.21.15.106:0/3277518210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700b6170 0x7f75700c8c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f757006ca80 tx=0x7f755802a660 comp rx=0 tx=0).stop 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 -- 172.21.15.106:0/3277518210 shutdown_connections 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 --2- 172.21.15.106:0/3277518210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700b6170 0x7f75700c8c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 --2- 172.21.15.106:0/3277518210 >> v2:172.21.15.204:3300/0 conn(0x7f75700d19c0 0x7f75700d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 -- 172.21.15.106:0/3277518210 >> 172.21.15.106:0/3277518210 conn(0x7f757009e460 msgr2=0x7f75700d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 -- 172.21.15.106:0/3277518210 shutdown_connections 2024-04-26T21:46:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.512+0000 7f7575400640 1 -- 172.21.15.106:0/3277518210 wait complete. 2024-04-26T21:46:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.514+0000 7f7575400640 1 Processor -- start 2024-04-26T21:46:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.514+0000 7f7575400640 1 -- start start 2024-04-26T21:46:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b6170 0x7f75700b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700d19c0 0x7f757009edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75700cfc30 con 0x7f75700d1da0 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75700c91b0 con 0x7f75700d19c0 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f7575400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7570162d50 con 0x7f75700b6170 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f756f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f756f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46768/0 (socket says 172.21.15.106:46768) 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.515+0000 7f756f200640 1 -- 172.21.15.106:0/3866163369 learned_addr learned my addr 172.21.15.106:0/3866163369 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756de00640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700d19c0 0x7f757009edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756e800640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b6170 0x7f75700b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 -- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b6170 msgr2=0x7f75700b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b6170 0x7f75700b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 -- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700d19c0 msgr2=0x7f757009edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700d19c0 0x7f757009edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 -- 172.21.15.106:0/3866163369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75700b3000 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f756f200640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f756800b410 tx=0x7f756800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568015030 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568004020 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568004410 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.516+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75700b2d20 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.517+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75700a1130 con 0x7f75700d1da0 2024-04-26T21:46:29.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.518+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75700a0ce0 con 0x7f75700d1da0 2024-04-26T21:46:29.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.519+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7568018070 con 0x7f75700d1da0 2024-04-26T21:46:29.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.520+0000 7f7566a00640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 0x7f754c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.520+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f756808b6e0 con 0x7f75700d1da0 2024-04-26T21:46:29.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.520+0000 7f756e800640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 0x7f754c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.521+0000 7f756e800640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 0x7f754c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7560006fd0 tx=0x7f7560008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.525+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7568059a70 con 0x7f75700d1da0 2024-04-26T21:46:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.721+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75700d26d0 con 0x7f75700d1da0 2024-04-26T21:46:29.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.722+0000 7f7566a00640 1 -- 172.21.15.106:0/3866163369 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7568059370 con 0x7f75700d1da0 2024-04-26T21:46:29.723 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 msgr2=0x7f754c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 0x7f754c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7560006fd0 tx=0x7f7560008040 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 msgr2=0x7f757009f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f756800b410 tx=0x7f756800b8d0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 shutdown_connections 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75700d1da0 0x7f757009f310 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700d19c0 0x7f757009edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f754c06c8f0 0x7f754c06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 --2- 172.21.15.106:0/3866163369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b6170 0x7f75700b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.724+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 >> 172.21.15.106:0/3866163369 conn(0x7f757009e460 msgr2=0x7f75700d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.725+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 shutdown_connections 2024-04-26T21:46:29.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.725+0000 7f7575400640 1 -- 172.21.15.106:0/3866163369 wait complete. 2024-04-26T21:46:29.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.934+0000 7f9470e00640 1 Processor -- start 2024-04-26T21:46:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.935+0000 7f9470e00640 1 -- start start 2024-04-26T21:46:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.936+0000 7f9470e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c160d60 0x7f946c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.936+0000 7f9470e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f946c146280 0x7f946c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.936+0000 7f9470e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f946c0b4d20 con 0x7f946c161140 2024-04-26T21:46:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.936+0000 7f9470e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f946c06e5f0 con 0x7f946c146280 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.936+0000 7f9470e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f946c146c30 con 0x7f946c160d60 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f946c146280 0x7f946c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f946c146280 0x7f946c1466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46776/0 (socket says 172.21.15.106:46776) 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 -- 172.21.15.106:0/1438337016 learned_addr learned my addr 172.21.15.106:0/1438337016 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946b400640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c160d60 0x7f946c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 -- 172.21.15.106:0/1438337016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c160d60 msgr2=0x7f946c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c160d60 0x7f946c0b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 -- 172.21.15.106:0/1438337016 >> v1:172.21.15.204:6789/0 conn(0x7f946c161140 legacy=0x7f946c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.937+0000 7f946be00640 1 -- 172.21.15.106:0/1438337016 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c0b5000 con 0x7f946c146280 2024-04-26T21:46:29.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.938+0000 7f946be00640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c146280 0x7f946c1466f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9464009e50 tx=0x7f946402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c3a4ad27f4f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f946aa00640 1 -- 172.21.15.106:0/1438337016 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464036070 con 0x7f946c146280 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f946aa00640 1 -- 172.21.15.106:0/1438337016 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9464031600 con 0x7f946c146280 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f946aa00640 1 -- 172.21.15.106:0/1438337016 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640319d0 con 0x7f946c146280 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 -- 172.21.15.106:0/1438337016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c146280 msgr2=0x7f946c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c146280 0x7f946c1466f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9464009e50 tx=0x7f946402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 -- 172.21.15.106:0/1438337016 shutdown_connections 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c146280 0x7f946c1466f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 --2- 172.21.15.106:0/1438337016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c160d60 0x7f946c0b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.939+0000 7f9470e00640 1 -- 172.21.15.106:0/1438337016 >> 172.21.15.106:0/1438337016 conn(0x7f946c0b2900 msgr2=0x7f946c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.940+0000 7f9470e00640 1 -- 172.21.15.106:0/1438337016 shutdown_connections 2024-04-26T21:46:29.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.940+0000 7f9470e00640 1 -- 172.21.15.106:0/1438337016 wait complete. 2024-04-26T21:46:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.941+0000 7f9470e00640 1 Processor -- start 2024-04-26T21:46:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.941+0000 7f9470e00640 1 -- start start 2024-04-26T21:46:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c146280 0x7f946c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c161140 0x7f946c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f946c0cf6f0 con 0x7f946c160d60 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f946c0cf410 con 0x7f946c146280 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.942+0000 7f9470e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f946c0b6070 con 0x7f946c161140 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46782/0 (socket says 172.21.15.106:46782) 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 -- 172.21.15.106:0/3911100634 learned_addr learned my addr 172.21.15.106:0/3911100634 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f946b400640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c146280 0x7f946c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f946be00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c161140 0x7f946c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 -- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c161140 msgr2=0x7f946c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c161140 0x7f946c0cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 -- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c146280 msgr2=0x7f946c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c146280 0x7f946c158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 -- 172.21.15.106:0/3911100634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c15e060 con 0x7f946c160d60 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.943+0000 7f9463e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f945400c430 tx=0x7f945400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.944+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454011790 con 0x7f946c160d60 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.944+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9454017070 con 0x7f946c160d60 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.944+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454011bb0 con 0x7f946c160d60 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.944+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c13d8c0 con 0x7f946c160d60 2024-04-26T21:46:29.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.944+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c159240 con 0x7f946c160d60 2024-04-26T21:46:29.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.945+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946c13df80 con 0x7f946c160d60 2024-04-26T21:46:29.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.946+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f945401c460 con 0x7f946c160d60 2024-04-26T21:46:29.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.947+0000 7f9463400640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.947+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f945408b9e0 con 0x7f946c160d60 2024-04-26T21:46:29.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.947+0000 7f946b400640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.948+0000 7f946b400640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 0x7f944806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f945c006fd0 tx=0x7f945c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:29.951+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9454059ea0 con 0x7f946c160d60 2024-04-26T21:46:30.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.150+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f946c0693c0 con 0x7f946c160d60 2024-04-26T21:46:30.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.150+0000 7f9463400640 1 -- 172.21.15.106:0/3911100634 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f94540597a0 con 0x7f946c160d60 2024-04-26T21:46:30.151 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 msgr2=0x7f944806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 0x7f944806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f945c006fd0 tx=0x7f945c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 msgr2=0x7f946c15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f945400c430 tx=0x7f945400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 shutdown_connections 2024-04-26T21:46:30.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c161140 0x7f946c0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c160d60 0x7f946c15af00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 --2- 172.21.15.106:0/3911100634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c146280 0x7f946c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 >> 172.21.15.106:0/3911100634 conn(0x7f946c0b2900 msgr2=0x7f946c15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 shutdown_connections 2024-04-26T21:46:30.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.153+0000 7f9470e00640 1 -- 172.21.15.106:0/3911100634 wait complete. 2024-04-26T21:46:30.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.373+0000 7f1be9a00640 1 Processor -- start 2024-04-26T21:46:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.373+0000 7f1be9a00640 1 -- start start 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1be415d100 0x7f1be406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1be40b5ca0 0x7f1be406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be40add80 0x7f1be4072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1be40af800 con 0x7f1be40b5ca0 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1be40ae340 con 0x7f1be415d100 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1be40afaa0 con 0x7f1be40add80 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be2000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1be40b5ca0 0x7f1be406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be2000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1be40b5ca0 0x7f1be406ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46790/0 (socket says 172.21.15.106:46790) 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.375+0000 7f1be2000640 1 -- 172.21.15.106:0/365147557 learned_addr learned my addr 172.21.15.106:0/365147557 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2a00640 1 --2- 172.21.15.106:0/365147557 >> v2:172.21.15.204:3300/0 conn(0x7f1be415d100 0x7f1be406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be3400640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be40add80 0x7f1be4072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2000640 1 -- 172.21.15.106:0/365147557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be40add80 msgr2=0x7f1be4072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2000640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be40add80 0x7f1be4072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2000640 1 -- 172.21.15.106:0/365147557 >> v2:172.21.15.204:3300/0 conn(0x7f1be415d100 msgr2=0x7f1be406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2000640 1 --2- 172.21.15.106:0/365147557 >> v2:172.21.15.204:3300/0 conn(0x7f1be415d100 0x7f1be406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.376+0000 7f1be2000640 1 -- 172.21.15.106:0/365147557 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be40afd80 con 0x7f1be40b5ca0 2024-04-26T21:46:30.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.377+0000 7f1be2000640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40b5ca0 0x7f1be406ccb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1be40730d0 tx=0x7f1bd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28e581d343fb7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.377+0000 7f1be1600640 1 -- 172.21.15.106:0/365147557 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4037070 con 0x7f1be40b5ca0 2024-04-26T21:46:30.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.377+0000 7f1be1600640 1 -- 172.21.15.106:0/365147557 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd4032440 con 0x7f1be40b5ca0 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.377+0000 7f1be1600640 1 -- 172.21.15.106:0/365147557 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4032810 con 0x7f1be40b5ca0 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 -- 172.21.15.106:0/365147557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40b5ca0 msgr2=0x7f1be406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40b5ca0 0x7f1be406ccb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1be40730d0 tx=0x7f1bd402a140 comp rx=0 tx=0).stop 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 -- 172.21.15.106:0/365147557 shutdown_connections 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be40add80 0x7f1be4072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 --2- 172.21.15.106:0/365147557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40b5ca0 0x7f1be406ccb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 --2- 172.21.15.106:0/365147557 >> v2:172.21.15.204:3300/0 conn(0x7f1be415d100 0x7f1be406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 -- 172.21.15.106:0/365147557 >> 172.21.15.106:0/365147557 conn(0x7f1be40b2d50 msgr2=0x7f1be40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 -- 172.21.15.106:0/365147557 shutdown_connections 2024-04-26T21:46:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.378+0000 7f1be9a00640 1 -- 172.21.15.106:0/365147557 wait complete. 2024-04-26T21:46:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.380+0000 7f1be9a00640 1 Processor -- start 2024-04-26T21:46:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.380+0000 7f1be9a00640 1 -- start start 2024-04-26T21:46:30.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.381+0000 7f1be9a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be40b5ca0 0x7f1be40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be415d100 0x7f1be40b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be9a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1be409ee50 con 0x7f1be40add80 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1be40b0340 con 0x7f1be40b5ca0 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1be4160b70 con 0x7f1be415d100 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46798/0 (socket says 172.21.15.106:46798) 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 -- 172.21.15.106:0/3650956158 learned_addr learned my addr 172.21.15.106:0/3650956158 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2000640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be40b5ca0 0x7f1be40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be3400640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be415d100 0x7f1be40b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 -- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be415d100 msgr2=0x7f1be40b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be415d100 0x7f1be40b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 -- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be40b5ca0 msgr2=0x7f1be40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be40b5ca0 0x7f1be40b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.382+0000 7f1be2a00640 1 -- 172.21.15.106:0/3650956158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be406eb50 con 0x7f1be40add80 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1be2a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d620 tx=0x7f1bcc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00b620 con 0x7f1be40add80 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc019070 con 0x7f1be40add80 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be406edf0 con 0x7f1be40add80 2024-04-26T21:46:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00ba00 con 0x7f1be40add80 2024-04-26T21:46:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.383+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be406f3b0 con 0x7f1be40add80 2024-04-26T21:46:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.384+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be4069650 con 0x7f1be40add80 2024-04-26T21:46:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.385+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bcc00bba0 con 0x7f1be40add80 2024-04-26T21:46:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.386+0000 7f1bdaa00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 0x7f1bc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.386+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1bcc08bec0 con 0x7f1be40add80 2024-04-26T21:46:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.387+0000 7f1be2000640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 0x7f1bc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.387+0000 7f1be2000640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 0x7f1bc006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4034000 tx=0x7f1bd402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.391+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bcc05a2e0 con 0x7f1be40add80 2024-04-26T21:46:30.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.574+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1be40740b0 con 0x7f1be40add80 2024-04-26T21:46:30.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.575+0000 7f1bdaa00640 1 -- 172.21.15.106:0/3650956158 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1bcc059be0 con 0x7f1be40add80 2024-04-26T21:46:30.576 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 msgr2=0x7f1bc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 0x7f1bc006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4034000 tx=0x7f1bd402e000 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 msgr2=0x7f1be414de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d620 tx=0x7f1bcc00dae0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 shutdown_connections 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1bc006c8f0 0x7f1bc006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be415d100 0x7f1be40b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be40b5ca0 0x7f1be40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 --2- 172.21.15.106:0/3650956158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1be40add80 0x7f1be414de90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.577+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 >> 172.21.15.106:0/3650956158 conn(0x7f1be40b2d50 msgr2=0x7f1be40b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.578+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 shutdown_connections 2024-04-26T21:46:30.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.578+0000 7f1be9a00640 1 -- 172.21.15.106:0/3650956158 wait complete. 2024-04-26T21:46:30.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:30.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.781+0000 7f6066200640 1 Processor -- start 2024-04-26T21:46:30.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.781+0000 7f6066200640 1 -- start start 2024-04-26T21:46:30.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.782+0000 7f6066200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60600cd650 0x7f60600add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.782+0000 7f6066200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600af9a0 con 0x7f606015cdf0 2024-04-26T21:46:30.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.782+0000 7f6066200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600afc80 con 0x7f606006ce80 2024-04-26T21:46:30.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.782+0000 7f6066200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f60600ae2c0 con 0x7f60600cd650 2024-04-26T21:46:30.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.783+0000 7f605f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f606015cdf0 0x7f606006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43568/0 (socket says 172.21.15.106:43568) 2024-04-26T21:46:30.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.783+0000 7f605f400640 1 -- 172.21.15.106:0/799177753 learned_addr learned my addr 172.21.15.106:0/799177753 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.783+0000 7f605fe00640 1 --2- 172.21.15.106:0/799177753 >> v2:172.21.15.204:3300/0 conn(0x7f60600cd650 0x7f60600add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.783+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478693048 0 0) 0x7f60600af9a0 con 0x7f606015cdf0 2024-04-26T21:46:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.783+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6044003520 con 0x7f606015cdf0 2024-04-26T21:46:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137842783 0 0) 0x7f6044003520 con 0x7f606015cdf0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 >> v2:172.21.15.204:3300/0 conn(0x7f60600cd650 msgr2=0x7f60600add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 --2- 172.21.15.106:0/799177753 >> v2:172.21.15.204:3300/0 conn(0x7f60600cd650 0x7f60600add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 >> v1:172.21.15.204:6789/0 conn(0x7f606006ce80 legacy=0x7f60600b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600aff60 con 0x7f606015cdf0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f6050001670 con 0x7f606015cdf0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 >> v1:172.21.15.106:6789/0 conn(0x7f606015cdf0 legacy=0x7f606006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60440039b0 0x7f6044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6044004800 0x7f6044004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 0x7f604400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f604400c030 con 0x7f60440051c0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f604400c1e0 con 0x7f60440039b0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f604400c3e0 con 0x7f6044004800 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605fe00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 0x7f604400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605f400640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60440039b0 0x7f6044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.784+0000 7f605ea00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6044004800 0x7f6044004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 -- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6044004800 msgr2=0x7f6044004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6044004800 0x7f6044004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 -- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60440039b0 msgr2=0x7f6044003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60440039b0 0x7f6044003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 -- 172.21.15.106:0/799177753 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6044003520 con 0x7f60440051c0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605f400640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60440039b0 0x7f6044003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605ea00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6044004800 0x7f6044004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.785+0000 7f605fe00640 1 -- 172.21.15.106:0/799177753 reap_dead start 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f605fe00640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 0x7f604400b9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6058009a90 tx=0x7f6044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8900e545123063 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058036070 con 0x7f60440051c0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058031400 con 0x7f60440051c0 2024-04-26T21:46:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f605e000640 1 -- 172.21.15.106:0/799177753 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058009690 con 0x7f60440051c0 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f6066200640 1 -- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 msgr2=0x7f604400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f6066200640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 0x7f604400b9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6058009a90 tx=0x7f6044005830 comp rx=0 tx=0).stop 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f6066200640 1 -- 172.21.15.106:0/799177753 shutdown_connections 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f6066200640 1 --2- 172.21.15.106:0/799177753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60440051c0 0x7f604400b9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.786+0000 7f6066200640 1 -- 172.21.15.106:0/799177753 >> 172.21.15.106:0/799177753 conn(0x7f60600b2d50 msgr2=0x7f60600b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.787+0000 7f6066200640 1 -- 172.21.15.106:0/799177753 shutdown_connections 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.787+0000 7f6066200640 1 -- 172.21.15.106:0/799177753 wait complete. 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.788+0000 7f6066200640 1 Processor -- start 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.788+0000 7f6066200640 1 -- start start 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60440051c0 0x7f606006f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f606014f980 0x7f606009d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f606015d1d0 con 0x7f606009cc20 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60600b05b0 con 0x7f606014f980 2024-04-26T21:46:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f6066200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f606015d350 con 0x7f60440051c0 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f605ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f605ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46816/0 (socket says 172.21.15.106:46816) 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.789+0000 7f605ea00640 1 -- 172.21.15.106:0/1650644141 learned_addr learned my addr 172.21.15.106:0/1650644141 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605f400640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60440051c0 0x7f606006f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 -- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60440051c0 msgr2=0x7f606006f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60440051c0 0x7f606006f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 -- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f606014f980 msgr2=0x7f606009d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f606014f980 0x7f606009d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 -- 172.21.15.106:0/1650644141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606009bb80 con 0x7f606009cc20 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f605ea00640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f604c00d8b0 tx=0x7f604c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c003da0 con 0x7f606009cc20 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c017070 con 0x7f606009cc20 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c005290 con 0x7f606009cc20 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.790+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606009be60 con 0x7f606009cc20 2024-04-26T21:46:30.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.791+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606009c420 con 0x7f606009cc20 2024-04-26T21:46:30.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.792+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060069540 con 0x7f606009cc20 2024-04-26T21:46:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.793+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f604c005430 con 0x7f606009cc20 2024-04-26T21:46:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.793+0000 7f6057400640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.794+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f604c08bc80 con 0x7f606009cc20 2024-04-26T21:46:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.794+0000 7f605f400640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.794+0000 7f605f400640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 0x7f604006ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6050001200 tx=0x7f6050009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.798+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f604c05a0a0 con 0x7f606009cc20 2024-04-26T21:46:30.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.997+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f606015c610 con 0x7f606009cc20 2024-04-26T21:46:31.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:30.999+0000 7f6057400640 1 -- 172.21.15.106:0/1650644141 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f604c0599a0 con 0x7f606009cc20 2024-04-26T21:46:31.001 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:31.001 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:31.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 msgr2=0x7f604006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 0x7f604006ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6050001200 tx=0x7f6050009040 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 msgr2=0x7f606014f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f604c00d8b0 tx=0x7f604c00dd70 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 shutdown_connections 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f606014f980 0x7f606009d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f606009cc20 0x7f606014f440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 --2- 172.21.15.106:0/1650644141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60440051c0 0x7f606006f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 >> 172.21.15.106:0/1650644141 conn(0x7f60600b2d50 msgr2=0x7f606006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 shutdown_connections 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.002+0000 7f6066200640 1 -- 172.21.15.106:0/1650644141 wait complete. 2024-04-26T21:46:31.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:31.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.218+0000 7f1edfa00640 1 Processor -- start 2024-04-26T21:46:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.218+0000 7f1edfa00640 1 -- start start 2024-04-26T21:46:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edfa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ed80d1da0 0x7f1ed8162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edfa00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed80d3830 con 0x7f1ed80d19c0 2024-04-26T21:46:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edfa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed80d3b10 con 0x7f1ed80b6170 2024-04-26T21:46:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edfa00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed80d40d0 con 0x7f1ed80d1da0 2024-04-26T21:46:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edda00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ed80d1da0 0x7f1ed8162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edd000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1ed80d19c0 0x7f1ed80d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43572/0 (socket says 172.21.15.106:43572) 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.220+0000 7f1edd000640 1 -- 172.21.15.106:0/3614797795 learned_addr learned my addr 172.21.15.106:0/3614797795 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.221+0000 7f1edda00640 1 -- 172.21.15.106:0/3614797795 >> v1:172.21.15.204:6789/0 conn(0x7f1ed80b6170 legacy=0x7f1ed80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.221+0000 7f1edda00640 1 -- 172.21.15.106:0/3614797795 >> v1:172.21.15.106:6789/0 conn(0x7f1ed80d19c0 legacy=0x7f1ed80d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.221+0000 7f1edda00640 1 -- 172.21.15.106:0/3614797795 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80d3df0 con 0x7f1ed80d1da0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.222+0000 7f1edda00640 1 --2- 172.21.15.106:0/3614797795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d1da0 0x7f1ed8162640 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00097a0 tx=0x7f1ed002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2673290d0b4c2f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.222+0000 7f1ed7400640 1 -- 172.21.15.106:0/3614797795 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0037070 con 0x7f1ed80d1da0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.222+0000 7f1ed7400640 1 -- 172.21.15.106:0/3614797795 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed002ae50 con 0x7f1ed80d1da0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.222+0000 7f1ed7400640 1 -- 172.21.15.106:0/3614797795 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0032660 con 0x7f1ed80d1da0 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 -- 172.21.15.106:0/3614797795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d1da0 msgr2=0x7f1ed8162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 --2- 172.21.15.106:0/3614797795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d1da0 0x7f1ed8162640 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00097a0 tx=0x7f1ed002a140 comp rx=0 tx=0).stop 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 -- 172.21.15.106:0/3614797795 shutdown_connections 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 --2- 172.21.15.106:0/3614797795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d1da0 0x7f1ed8162640 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 -- 172.21.15.106:0/3614797795 >> 172.21.15.106:0/3614797795 conn(0x7f1ed809e460 msgr2=0x7f1ed80d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 -- 172.21.15.106:0/3614797795 shutdown_connections 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.223+0000 7f1edfa00640 1 -- 172.21.15.106:0/3614797795 wait complete. 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.225+0000 7f1edfa00640 1 Processor -- start 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.225+0000 7f1edfa00640 1 -- start start 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ed80b6170 0x7f1ed80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ed80d1da0 0x7f1ed806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8162d00 con 0x7f1ed80d19c0 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8162b80 con 0x7f1ed80b6170 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed8162e80 con 0x7f1ed80d1da0 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46834/0 (socket says 172.21.15.106:46834) 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 -- 172.21.15.106:0/487029618 learned_addr learned my addr 172.21.15.106:0/487029618 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 -- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ed80d1da0 msgr2=0x7f1ed806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ed80d1da0 0x7f1ed806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 -- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ed80b6170 msgr2=0x7f1ed80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1edd000640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ed80b6170 0x7f1ed80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ed80b6170 0x7f1ed80b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.226+0000 7f1ed7e00640 1 -- 172.21.15.106:0/487029618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80b3dd0 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1ed7e00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ba50 tx=0x7f1ec000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0019070 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec000cba0 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0014440 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed81f5c80 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.227+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed81f6240 con 0x7f1ed80d19c0 2024-04-26T21:46:31.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.228+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed8162110 con 0x7f1ed80d19c0 2024-04-26T21:46:31.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.229+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec001c070 con 0x7f1ed80d19c0 2024-04-26T21:46:31.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.231+0000 7f1ed4c00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.231+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ec008b700 con 0x7f1ed80d19c0 2024-04-26T21:46:31.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.231+0000 7f1edd000640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.231+0000 7f1edd000640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 0x7f1eb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000f60 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.235+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ed8162110 con 0x7f1ed80d19c0 2024-04-26T21:46:31.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.434+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ed809e8e0 con 0x7f1ed80d19c0 2024-04-26T21:46:31.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.434+0000 7f1ed4c00640 1 -- 172.21.15.106:0/487029618 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1ed809e8e0 con 0x7f1ed80d19c0 2024-04-26T21:46:31.435 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.436+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 msgr2=0x7f1eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.436+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 0x7f1eb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000f60 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 msgr2=0x7f1ed806e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ba50 tx=0x7f1ec000bf10 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 shutdown_connections 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ed80d1da0 0x7f1ed806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ed80d19c0 0x7f1ed806e170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 --2- 172.21.15.106:0/487029618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ed80b6170 0x7f1ed80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 >> 172.21.15.106:0/487029618 conn(0x7f1ed809e460 msgr2=0x7f1ed81601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 shutdown_connections 2024-04-26T21:46:31.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.437+0000 7f1edfa00640 1 -- 172.21.15.106:0/487029618 wait complete. 2024-04-26T21:46:31.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:31.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.661+0000 7f8693400640 1 Processor -- start 2024-04-26T21:46:31.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.661+0000 7f8693400640 1 -- start start 2024-04-26T21:46:31.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8693400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f868c06c770 0x7f868c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8693400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f868c0b3c30 0x7f868c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8693400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c06eea0 con 0x7f868c156650 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8693400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f868c06f140 con 0x7f868c06c770 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8693400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f868c06f420 con 0x7f868c0b3c30 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8683e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f868c06c770 0x7f868c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8683e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f868c06c770 0x7f868c06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46840/0 (socket says 172.21.15.106:46840) 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.663+0000 7f8683e00640 1 -- 172.21.15.106:0/1875395534 learned_addr learned my addr 172.21.15.106:0/1875395534 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.664+0000 7f8691400640 1 --2- 172.21.15.106:0/1875395534 >> v2:172.21.15.204:3300/0 conn(0x7f868c0b3c30 0x7f868c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.664+0000 7f8683e00640 1 -- 172.21.15.106:0/1875395534 >> v2:172.21.15.204:3300/0 conn(0x7f868c0b3c30 msgr2=0x7f868c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.664+0000 7f8683e00640 1 --2- 172.21.15.106:0/1875395534 >> v2:172.21.15.204:3300/0 conn(0x7f868c0b3c30 0x7f868c0b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.664+0000 7f8683e00640 1 -- 172.21.15.106:0/1875395534 >> v1:172.21.15.204:6789/0 conn(0x7f868c156650 legacy=0x7f868c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.664+0000 7f8683e00640 1 -- 172.21.15.106:0/1875395534 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c06f700 con 0x7f868c06c770 2024-04-26T21:46:31.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.665+0000 7f8683e00640 1 --2- 172.21.15.106:0/1875395534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c06c770 0x7f868c06cbf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f868c0d0c70 tx=0x7f867402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca33f0750737ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.665+0000 7f868be00640 1 -- 172.21.15.106:0/1875395534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674037070 con 0x7f868c06c770 2024-04-26T21:46:31.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.665+0000 7f868be00640 1 -- 172.21.15.106:0/1875395534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8674032440 con 0x7f868c06c770 2024-04-26T21:46:31.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.665+0000 7f868be00640 1 -- 172.21.15.106:0/1875395534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674032810 con 0x7f868c06c770 2024-04-26T21:46:31.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 -- 172.21.15.106:0/1875395534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c06c770 msgr2=0x7f868c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 --2- 172.21.15.106:0/1875395534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c06c770 0x7f868c06cbf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f868c0d0c70 tx=0x7f867402a230 comp rx=0 tx=0).stop 2024-04-26T21:46:31.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 -- 172.21.15.106:0/1875395534 shutdown_connections 2024-04-26T21:46:31.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 --2- 172.21.15.106:0/1875395534 >> v2:172.21.15.204:3300/0 conn(0x7f868c0b3c30 0x7f868c0b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 --2- 172.21.15.106:0/1875395534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c06c770 0x7f868c06cbf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 -- 172.21.15.106:0/1875395534 >> 172.21.15.106:0/1875395534 conn(0x7f868c0b2d50 msgr2=0x7f868c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 -- 172.21.15.106:0/1875395534 shutdown_connections 2024-04-26T21:46:31.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.666+0000 7f8693400640 1 -- 172.21.15.106:0/1875395534 wait complete. 2024-04-26T21:46:31.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.668+0000 7f8693400640 1 Processor -- start 2024-04-26T21:46:31.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.669+0000 7f8693400640 1 -- start start 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868c06c770 0x7f868c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c156650 0x7f868c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f868c15c680 con 0x7f868c0b3c30 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f868c06fcc0 con 0x7f868c156650 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8693400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f868c14db50 con 0x7f868c06c770 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46852/0 (socket says 172.21.15.106:46852) 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 -- 172.21.15.106:0/4103301150 learned_addr learned my addr 172.21.15.106:0/4103301150 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:31.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8691400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c156650 0x7f868c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 -- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868c06c770 msgr2=0x7f868c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868c06c770 0x7f868c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 -- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c156650 msgr2=0x7f868c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c156650 0x7f868c159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.670+0000 7f8683e00640 1 -- 172.21.15.106:0/4103301150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c14dda0 con 0x7f868c0b3c30 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8683e00640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8674000c00 tx=0x7f86740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674041020 con 0x7f868c0b3c30 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86740306c0 con 0x7f868c0b3c30 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674037040 con 0x7f868c0b3c30 2024-04-26T21:46:31.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c14e080 con 0x7f868c0b3c30 2024-04-26T21:46:31.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.671+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c0ae990 con 0x7f868c0b3c30 2024-04-26T21:46:31.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.673+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c069650 con 0x7f868c0b3c30 2024-04-26T21:46:31.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.673+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8674044020 con 0x7f868c0b3c30 2024-04-26T21:46:31.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.675+0000 7f8689600640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 0x7f866806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.675+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86740ac3e0 con 0x7f868c0b3c30 2024-04-26T21:46:31.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.675+0000 7f8690a00640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 0x7f866806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.676+0000 7f8690a00640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 0x7f866806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f867c006fd0 tx=0x7f867c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.679+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f867404b030 con 0x7f868c0b3c30 2024-04-26T21:46:31.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.879+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f868c15cbc0 con 0x7f868c0b3c30 2024-04-26T21:46:31.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.880+0000 7f8689600640 1 -- 172.21.15.106:0/4103301150 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f867407a800 con 0x7f868c0b3c30 2024-04-26T21:46:31.881 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:31.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 msgr2=0x7f866806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 0x7f866806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f867c006fd0 tx=0x7f867c006bc0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 msgr2=0x7f868c073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8674000c00 tx=0x7f86740046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 shutdown_connections 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c156650 0x7f868c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c0b3c30 0x7f868c073570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f866806c8f0 0x7f866806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 --2- 172.21.15.106:0/4103301150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f868c06c770 0x7f868c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 >> 172.21.15.106:0/4103301150 conn(0x7f868c0b2d50 msgr2=0x7f868c15e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 shutdown_connections 2024-04-26T21:46:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:31.883+0000 7f8693400640 1 -- 172.21.15.106:0/4103301150 wait complete. 2024-04-26T21:46:31.903 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.102+0000 7f2125e00640 1 Processor -- start 2024-04-26T21:46:32.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.102+0000 7f2125e00640 1 -- start start 2024-04-26T21:46:32.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.104+0000 7f2125e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21200b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.104+0000 7f2125e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2120073da0 0x7f212006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.104+0000 7f2125e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21200afed0 con 0x7f212006ccb0 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.104+0000 7f2125e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f212006d0e0 con 0x7f2120073da0 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.104+0000 7f2125e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f21200ade20 con 0x7f2120147f20 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f211f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f212006ccb0 0x7f2120072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43582/0 (socket says 172.21.15.106:43582) 2024-04-26T21:46:32.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f211f400640 1 -- 172.21.15.106:0/3620855188 learned_addr learned my addr 172.21.15.106:0/3620855188 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f211ea00640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21200b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f2116000640 1 --2- 172.21.15.106:0/3620855188 >> v2:172.21.15.204:3300/0 conn(0x7f2120073da0 0x7f212006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849946518 0 0) 0x7f2118001490 con 0x7f212006ccb0 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.105+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2104003520 con 0x7f212006ccb0 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393100050 0 0) 0x7f2104003520 con 0x7f212006ccb0 2024-04-26T21:46:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 msgr2=0x7f21200b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21200b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 >> v2:172.21.15.204:3300/0 conn(0x7f2120073da0 msgr2=0x7f212006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 --2- 172.21.15.106:0/3620855188 >> v2:172.21.15.204:3300/0 conn(0x7f2120073da0 0x7f212006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200ae100 con 0x7f212006ccb0 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2118001bc0 con 0x7f212006ccb0 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 >> v1:172.21.15.106:6789/0 conn(0x7f212006ccb0 legacy=0x7f2120072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21040039b0 0x7f2104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2104004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f212006d0e0 con 0x7f2104004800 2024-04-26T21:46:32.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21200ade20 con 0x7f21040039b0 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f210400b9c0 con 0x7f2120147f20 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.106+0000 7f2116000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2104004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f2116000640 1 -- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 msgr2=0x7f21040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f211f400640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f211ea00640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21040039b0 0x7f2104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f2116000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2120147f20 0x7f21040051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f2116000640 1 -- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21040039b0 msgr2=0x7f2104003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f2116000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21040039b0 0x7f2104003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.107+0000 7f2116000640 1 -- 172.21.15.106:0/3620855188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2104003520 con 0x7f2104004800 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.108+0000 7f2116000640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2104004c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2108009ac0 tx=0x7f2104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f37ef3a25967816 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.108+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108036070 con 0x7f2104004800 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.108+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2108031400 con 0x7f2104004800 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.108+0000 7f211e000640 1 -- 172.21.15.106:0/3620855188 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108009690 con 0x7f2104004800 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 -- 172.21.15.106:0/3620855188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 msgr2=0x7f2104004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 --2- 172.21.15.106:0/3620855188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2104004c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2108009ac0 tx=0x7f2104005830 comp rx=0 tx=0).stop 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f211f400640 1 -- 172.21.15.106:0/3620855188 reap_dead start 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 -- 172.21.15.106:0/3620855188 shutdown_connections 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 -- 172.21.15.106:0/3620855188 >> 172.21.15.106:0/3620855188 conn(0x7f21200b2d50 msgr2=0x7f21200b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 -- 172.21.15.106:0/3620855188 shutdown_connections 2024-04-26T21:46:32.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.109+0000 7f2125e00640 1 -- 172.21.15.106:0/3620855188 wait complete. 2024-04-26T21:46:32.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.111+0000 7f2125e00640 1 Processor -- start 2024-04-26T21:46:32.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.111+0000 7f2125e00640 1 -- start start 2024-04-26T21:46:32.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 0x7f2120160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2120144050 0x7f21201444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21200ae9a0 con 0x7f2104004800 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21200ae6c0 con 0x7f2120144050 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2125e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2120074200 con 0x7f212009c950 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46878/0 (socket says 172.21.15.106:46878) 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 -- 172.21.15.106:0/3518950602 learned_addr learned my addr 172.21.15.106:0/3518950602 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2116000640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 0x7f2120160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211f400640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2120144050 0x7f21201444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 -- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 msgr2=0x7f2120160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 0x7f2120160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 -- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2120144050 msgr2=0x7f21201444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2120144050 0x7f21201444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f211ea00640 1 -- 172.21.15.106:0/3518950602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212015f7e0 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.112+0000 7f2116000640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 0x7f2120160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f211ea00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2110010860 tx=0x7f2110010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000f040 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211000eb60 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211001c400 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212015faa0 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.113+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21200ceaa0 con 0x7f2104004800 2024-04-26T21:46:32.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.114+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2120069650 con 0x7f2104004800 2024-04-26T21:46:32.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.115+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f211001c5a0 con 0x7f2104004800 2024-04-26T21:46:32.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.116+0000 7f2117e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.117+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f211008c2e0 con 0x7f2104004800 2024-04-26T21:46:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.117+0000 7f2116000640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.117+0000 7f2116000640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2108033000 tx=0x7f2108008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.121+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2110018030 con 0x7f2104004800 2024-04-26T21:46:32.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.318+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2120144a20 con 0x7f2104004800 2024-04-26T21:46:32.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.319+0000 7f2117e00640 1 -- 172.21.15.106:0/3518950602 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f211005a020 con 0x7f2104004800 2024-04-26T21:46:32.321 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:32.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 msgr2=0x7f20fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2108033000 tx=0x7f2108008030 comp rx=0 tx=0).stop 2024-04-26T21:46:32.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 msgr2=0x7f2120162050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2110010860 tx=0x7f2110010d20 comp rx=0 tx=0).stop 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 shutdown_connections 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2120144050 0x7f21201444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f212009c950 0x7f2120160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 --2- 172.21.15.106:0/3518950602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104004800 0x7f2120162050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 >> 172.21.15.106:0/3518950602 conn(0x7f21200b2d50 msgr2=0x7f21200b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 shutdown_connections 2024-04-26T21:46:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.322+0000 7f2125e00640 1 -- 172.21.15.106:0/3518950602 wait complete. 2024-04-26T21:46:32.341 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:32.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.539+0000 7f9ba3200640 1 Processor -- start 2024-04-26T21:46:32.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.539+0000 7f9ba3200640 1 -- start start 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba3200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba3200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9c15f800 con 0x7f9b9c15f420 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba3200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c0d2c70 con 0x7f9b9c0d3680 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c0d3480 con 0x7f9b9c0c9520 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46884/0 (socket says 172.21.15.106:46884) 2024-04-26T21:46:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 -- 172.21.15.106:0/2099741460 learned_addr learned my addr 172.21.15.106:0/2099741460 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9ba1200640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 -- 172.21.15.106:0/2099741460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 msgr2=0x7f9b9c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.541+0000 7f9b9b400640 1 -- 172.21.15.106:0/2099741460 >> v1:172.21.15.204:6789/0 conn(0x7f9b9c15f420 legacy=0x7f9b9c0d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.542+0000 7f9b9b400640 1 -- 172.21.15.106:0/2099741460 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0d4020 con 0x7f9b9c0d3680 2024-04-26T21:46:32.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.542+0000 7f9ba1200640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:32.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.543+0000 7f9b9b400640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b8402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e53e38e31b31a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.543+0000 7f9b9aa00640 1 -- 172.21.15.106:0/2099741460 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84037070 con 0x7f9b9c0d3680 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.543+0000 7f9b9aa00640 1 -- 172.21.15.106:0/2099741460 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84032440 con 0x7f9b9c0d3680 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.543+0000 7f9b9aa00640 1 -- 172.21.15.106:0/2099741460 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84032810 con 0x7f9b9c0d3680 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.543+0000 7f9ba3200640 1 -- 172.21.15.106:0/2099741460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c0d3680 msgr2=0x7f9b9c0d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b8402a180 comp rx=0 tx=0).stop 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 -- 172.21.15.106:0/2099741460 shutdown_connections 2024-04-26T21:46:32.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 --2- 172.21.15.106:0/2099741460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0d3ae0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 -- 172.21.15.106:0/2099741460 >> 172.21.15.106:0/2099741460 conn(0x7f9b9c09e3e0 msgr2=0x7f9b9c0d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 -- 172.21.15.106:0/2099741460 shutdown_connections 2024-04-26T21:46:32.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.544+0000 7f9ba3200640 1 -- 172.21.15.106:0/2099741460 wait complete. 2024-04-26T21:46:32.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.546+0000 7f9ba3200640 1 Processor -- start 2024-04-26T21:46:32.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.546+0000 7f9ba3200640 1 -- start start 2024-04-26T21:46:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0c8d20 con 0x7f9b9c15f420 2024-04-26T21:46:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0d31a0 con 0x7f9b9c0d3680 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c0c9000 con 0x7f9b9c0c9520 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46892/0 (socket says 172.21.15.106:46892) 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9ba1200640 1 -- 172.21.15.106:0/210262120 learned_addr learned my addr 172.21.15.106:0/210262120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.547+0000 7f9b9be00640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9b9b400640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 -- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 msgr2=0x7f9b9c1f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c1f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 -- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c0d3680 msgr2=0x7f9b9c0b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 -- 172.21.15.106:0/210262120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0ba380 con 0x7f9b9c15f420 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba1200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400cd50 tx=0x7f9b9400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400f680 con 0x7f9b9c15f420 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9400f840 con 0x7f9b9c15f420 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400fc30 con 0x7f9b9c15f420 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c0ba660 con 0x7f9b9c15f420 2024-04-26T21:46:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.548+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c0b7ed0 con 0x7f9b9c15f420 2024-04-26T21:46:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.550+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c069540 con 0x7f9b9c15f420 2024-04-26T21:46:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.550+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b940213f0 con 0x7f9b9c15f420 2024-04-26T21:46:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.551+0000 7f9b93e00640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 0x7f9b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.551+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b9408ca00 con 0x7f9b9c15f420 2024-04-26T21:46:32.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.552+0000 7f9b9be00640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 0x7f9b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.552+0000 7f9b9be00640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 0x7f9b7806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c004ea0 tx=0x7f9b8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.556+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b94014030 con 0x7f9b9c15f420 2024-04-26T21:46:32.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.739+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b9c0ca230 con 0x7f9b9c15f420 2024-04-26T21:46:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.740+0000 7f9b93e00640 1 -- 172.21.15.106:0/210262120 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f9b9405ad90 con 0x7f9b9c15f420 2024-04-26T21:46:32.742 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:32.742 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 msgr2=0x7f9b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 0x7f9b7806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c004ea0 tx=0x7f9b8c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 msgr2=0x7f9b9c14cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400cd50 tx=0x7f9b9400a650 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 shutdown_connections 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b9c15f420 0x7f9b9c14cf90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c0d3680 0x7f9b9c0b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9b7806c8f0 0x7f9b7806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 --2- 172.21.15.106:0/210262120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b9c0c9520 0x7f9b9c1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 >> 172.21.15.106:0/210262120 conn(0x7f9b9c09e3e0 msgr2=0x7f9b9c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.743+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 shutdown_connections 2024-04-26T21:46:32.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.744+0000 7f9ba3200640 1 -- 172.21.15.106:0/210262120 wait complete. 2024-04-26T21:46:32.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:32.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.962+0000 7f21aea00640 1 Processor -- start 2024-04-26T21:46:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.962+0000 7f21aea00640 1 -- start start 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.963+0000 7f21aea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21a8161140 0x7f21a806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.963+0000 7f21aea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8146280 0x7f21a81466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.963+0000 7f21aea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a80b4d20 con 0x7f21a8160d60 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.963+0000 7f21aea00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f21a8146c30 con 0x7f21a8161140 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.963+0000 7f21aea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f21a80b5000 con 0x7f21a8146280 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21ad600640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f21a8160d60 0x7f21a80b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43594/0 (socket says 172.21.15.106:43594) 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21ad600640 1 -- 172.21.15.106:0/1448593438 learned_addr learned my addr 172.21.15.106:0/1448593438 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21acc00640 1 --2- 172.21.15.106:0/1448593438 >> v2:172.21.15.106:3300/0 conn(0x7f21a8161140 0x7f21a806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21a7e00640 1 -- 172.21.15.106:0/1448593438 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934199644 0 0) 0x7f21a80b4d20 con 0x7f21a8160d60 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21ae000640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8146280 0x7f21a81466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21a7e00640 1 -- 172.21.15.106:0/1448593438 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2190003520 con 0x7f21a8160d60 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21acc00640 1 -- 172.21.15.106:0/1448593438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8146280 msgr2=0x7f21a81466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21acc00640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8146280 0x7f21a81466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21acc00640 1 -- 172.21.15.106:0/1448593438 >> v1:172.21.15.106:6789/0 conn(0x7f21a8160d60 legacy=0x7f21a80b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.964+0000 7f21acc00640 1 -- 172.21.15.106:0/1448593438 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a80b52e0 con 0x7f21a8161140 2024-04-26T21:46:32.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.965+0000 7f21acc00640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8161140 0x7f21a806dee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21940097d0 tx=0x7f219402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e30e49c1f20467d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21a7e00640 1 -- 172.21.15.106:0/1448593438 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21a8161140 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21a7e00640 1 -- 172.21.15.106:0/1448593438 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194032440 con 0x7f21a8161140 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21a7e00640 1 -- 172.21.15.106:0/1448593438 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194032810 con 0x7f21a8161140 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 -- 172.21.15.106:0/1448593438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8161140 msgr2=0x7f21a806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8161140 0x7f21a806dee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21940097d0 tx=0x7f219402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 -- 172.21.15.106:0/1448593438 shutdown_connections 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8146280 0x7f21a81466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 --2- 172.21.15.106:0/1448593438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8161140 0x7f21a806dee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.966+0000 7f21aea00640 1 -- 172.21.15.106:0/1448593438 >> 172.21.15.106:0/1448593438 conn(0x7f21a80b2900 msgr2=0x7f21a80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.967+0000 7f21aea00640 1 -- 172.21.15.106:0/1448593438 shutdown_connections 2024-04-26T21:46:32.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.967+0000 7f21aea00640 1 -- 172.21.15.106:0/1448593438 wait complete. 2024-04-26T21:46:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.968+0000 7f21aea00640 1 Processor -- start 2024-04-26T21:46:32.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.968+0000 7f21aea00640 1 -- start start 2024-04-26T21:46:32.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a8160d60 0x7f21a8140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8161140 0x7f21a8141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21a806e420 con 0x7f21a8146280 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21a80cf6f0 con 0x7f21a8160d60 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.969+0000 7f21aea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f21a806e5a0 con 0x7f21a8161140 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46904/0 (socket says 172.21.15.106:46904) 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 -- 172.21.15.106:0/2540011285 learned_addr learned my addr 172.21.15.106:0/2540011285 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 -- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8161140 msgr2=0x7f21a8141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8161140 0x7f21a8141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 -- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a8160d60 msgr2=0x7f21a8140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a8160d60 0x7f21a8140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 -- 172.21.15.106:0/2540011285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a8159d70 con 0x7f21a8146280 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.970+0000 7f21ad600640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f219c00cb40 tx=0x7f219c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.971+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c017070 con 0x7f21a8146280 2024-04-26T21:46:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.971+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f219c00aca0 con 0x7f21a8146280 2024-04-26T21:46:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.971+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c012620 con 0x7f21a8146280 2024-04-26T21:46:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.971+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a8072e20 con 0x7f21a8146280 2024-04-26T21:46:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.971+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a80733e0 con 0x7f21a8146280 2024-04-26T21:46:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.972+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a80693c0 con 0x7f21a8146280 2024-04-26T21:46:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.973+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f219c01a070 con 0x7f21a8146280 2024-04-26T21:46:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.974+0000 7f21a5600640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 0x7f218806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.974+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f219c08bab0 con 0x7f21a8146280 2024-04-26T21:46:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.974+0000 7f21acc00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 0x7f218806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.975+0000 7f21acc00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 0x7f218806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2194034000 tx=0x7f219402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:32.979+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21a8159d70 con 0x7f21a8146280 2024-04-26T21:46:33.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.177+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21a80d0530 con 0x7f21a8146280 2024-04-26T21:46:33.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.178+0000 7f21a5600640 1 -- 172.21.15.106:0/2540011285 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f219c059e40 con 0x7f21a8146280 2024-04-26T21:46:33.179 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 msgr2=0x7f218806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 0x7f218806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2194034000 tx=0x7f219402c000 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 msgr2=0x7f21a8159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f219c00cb40 tx=0x7f219c00a060 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 shutdown_connections 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f218806c8f0 0x7f218806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21a8161140 0x7f21a8141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a8160d60 0x7f21a8140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 --2- 172.21.15.106:0/2540011285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a8146280 0x7f21a8159830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.180+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 >> 172.21.15.106:0/2540011285 conn(0x7f21a80b2900 msgr2=0x7f21a815e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:33.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.181+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 shutdown_connections 2024-04-26T21:46:33.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.181+0000 7f21aea00640 1 -- 172.21.15.106:0/2540011285 wait complete. 2024-04-26T21:46:33.200 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.400+0000 7f4f2d400640 1 Processor -- start 2024-04-26T21:46:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.401+0000 7f4f2d400640 1 -- start start 2024-04-26T21:46:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f2d400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f280d19c0 0x7f4f280d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f2d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f280b6170 0x7f4f280c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f2d400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f280d3830 con 0x7f4f280d1da0 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f2d400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4f280d40d0 con 0x7f4f280d19c0 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f2d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4f280d3b10 con 0x7f4f280b6170 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f26800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f280d19c0 0x7f4f280d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f26800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f280d19c0 0x7f4f280d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46920/0 (socket says 172.21.15.106:46920) 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.402+0000 7f4f26800640 1 -- 172.21.15.106:0/1765514743 learned_addr learned my addr 172.21.15.106:0/1765514743 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.403+0000 7f4f26800640 1 -- 172.21.15.106:0/1765514743 >> v2:172.21.15.204:3300/0 conn(0x7f4f280b6170 msgr2=0x7f4f280c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.403+0000 7f4f26800640 1 --2- 172.21.15.106:0/1765514743 >> v2:172.21.15.204:3300/0 conn(0x7f4f280b6170 0x7f4f280c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.403+0000 7f4f26800640 1 -- 172.21.15.106:0/1765514743 >> v1:172.21.15.106:6789/0 conn(0x7f4f280d1da0 legacy=0x7f4f28162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.403+0000 7f4f26800640 1 -- 172.21.15.106:0/1765514743 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f280d3df0 con 0x7f4f280d19c0 2024-04-26T21:46:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.404+0000 7f4f26800640 1 --2- 172.21.15.106:0/1765514743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f280d2f00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f2808ede0 tx=0x7f4f1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ae2b42b27e4b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.404+0000 7f4f25400640 1 -- 172.21.15.106:0/1765514743 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18037070 con 0x7f4f280d19c0 2024-04-26T21:46:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.404+0000 7f4f25400640 1 -- 172.21.15.106:0/1765514743 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f18032440 con 0x7f4f280d19c0 2024-04-26T21:46:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.404+0000 7f4f25400640 1 -- 172.21.15.106:0/1765514743 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18032810 con 0x7f4f280d19c0 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 -- 172.21.15.106:0/1765514743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 msgr2=0x7f4f280d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1765514743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f280d2f00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f2808ede0 tx=0x7f4f1802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 -- 172.21.15.106:0/1765514743 shutdown_connections 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1765514743 >> v2:172.21.15.204:3300/0 conn(0x7f4f280b6170 0x7f4f280c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1765514743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f280d2f00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 -- 172.21.15.106:0/1765514743 >> 172.21.15.106:0/1765514743 conn(0x7f4f2809e460 msgr2=0x7f4f280d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:33.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 -- 172.21.15.106:0/1765514743 shutdown_connections 2024-04-26T21:46:33.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.405+0000 7f4f2d400640 1 -- 172.21.15.106:0/1765514743 wait complete. 2024-04-26T21:46:33.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.407+0000 7f4f2d400640 1 Processor -- start 2024-04-26T21:46:33.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.407+0000 7f4f2d400640 1 -- start start 2024-04-26T21:46:33.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 0x7f4f280ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f280d1da0 0x7f4f2809edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f280cfc30 con 0x7f4f280d19c0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f280c91b0 con 0x7f4f280b6170 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f2d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4f28162d50 con 0x7f4f280d1da0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f25e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f25e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46926/0 (socket says 172.21.15.106:46926) 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f25e00640 1 -- 172.21.15.106:0/1387535952 learned_addr learned my addr 172.21.15.106:0/1387535952 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f26800640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 0x7f4f280ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f27200640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f280d1da0 0x7f4f2809edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.408+0000 7f4f25e00640 1 -- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f280d1da0 msgr2=0x7f4f2809edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f25e00640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f280d1da0 0x7f4f2809edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f25e00640 1 -- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 msgr2=0x7f4f280ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f25e00640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 0x7f4f280ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f25e00640 1 -- 172.21.15.106:0/1387535952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f28157fb0 con 0x7f4f280d19c0 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f26800640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 0x7f4f280ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:33.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f25e00640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000ba50 tx=0x7f4f1000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10019070 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1000cc00 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10014440 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f28158290 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.409+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f28158850 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.410+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f28069650 con 0x7f4f280d19c0 2024-04-26T21:46:33.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.411+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f1001c070 con 0x7f4f280d19c0 2024-04-26T21:46:33.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.412+0000 7f4f1ea00640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 0x7f4f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.412+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f1008b980 con 0x7f4f280d19c0 2024-04-26T21:46:33.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.413+0000 7f4f26800640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 0x7f4f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.413+0000 7f4f26800640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 0x7f4f0406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f18034000 tx=0x7f4f1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.417+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f10059d10 con 0x7f4f280d19c0 2024-04-26T21:46:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.600+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f280cac80 con 0x7f4f280d19c0 2024-04-26T21:46:33.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.601+0000 7f4f1ea00640 1 -- 172.21.15.106:0/1387535952 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4f10059610 con 0x7f4f280d19c0 2024-04-26T21:46:33.602 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 msgr2=0x7f4f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 0x7f4f0406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f18034000 tx=0x7f4f1802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 msgr2=0x7f4f2806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000ba50 tx=0x7f4f1000bf10 comp rx=0 tx=0).stop 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 shutdown_connections 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4f0406c8f0 0x7f4f0406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f280d1da0 0x7f4f2809edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f280d19c0 0x7f4f2806e2a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 --2- 172.21.15.106:0/1387535952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f280b6170 0x7f4f280ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 >> 172.21.15.106:0/1387535952 conn(0x7f4f2809e460 msgr2=0x7f4f280c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 shutdown_connections 2024-04-26T21:46:33.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.605+0000 7f4f2d400640 1 -- 172.21.15.106:0/1387535952 wait complete. 2024-04-26T21:46:33.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:33.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.823+0000 7f5d41c00640 1 Processor -- start 2024-04-26T21:46:33.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.824+0000 7f5d41c00640 1 -- start start 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d3c1632e0 0x7f5d3c1636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 0x7f5d3c162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d3c0b6710 con 0x7f5d3c0b6210 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3c0d3600 con 0x7f5d3c1632e0 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c0d38e0 con 0x7f5d3c0d3110 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5d3c0b6210 0x7f5d3c0b4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43624/0 (socket says 172.21.15.106:43624) 2024-04-26T21:46:33.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d41200640 1 -- 172.21.15.106:0/3141029862 learned_addr learned my addr 172.21.15.106:0/3141029862 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d3be00640 1 --2- 172.21.15.106:0/3141029862 >> v2:172.21.15.204:3300/0 conn(0x7f5d3c1632e0 0x7f5d3c1636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.825+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189646682 0 0) 0x7f5d3c0b6710 con 0x7f5d3c0b6210 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d20003520 con 0x7f5d3c0b6210 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3b400640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 0x7f5d3c162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3126259659 0 0) 0x7f5d20003520 con 0x7f5d3c0b6210 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 msgr2=0x7f5d3c162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 0x7f5d3c162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 >> v2:172.21.15.204:3300/0 conn(0x7f5d3c1632e0 msgr2=0x7f5d3c1636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 --2- 172.21.15.106:0/3141029862 >> v2:172.21.15.204:3300/0 conn(0x7f5d3c1632e0 0x7f5d3c1636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c0d3bc0 con 0x7f5d3c0b6210 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f5d34001670 con 0x7f5d3c0b6210 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 >> v1:172.21.15.106:6789/0 conn(0x7f5d3c0b6210 legacy=0x7f5d3c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d20004800 0x7f5d20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 0x7f5d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c0d3600 con 0x7f5d20004800 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c0d38e0 con 0x7f5d200039b0 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2000b9f0 con 0x7f5d3c0d3110 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.826+0000 7f5d3b400640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d20004800 0x7f5d20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3b400640 1 -- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 msgr2=0x7f5d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:33.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3b400640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d3c0d3110 0x7f5d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3be00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3b400640 1 -- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d200039b0 msgr2=0x7f5d20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3b400640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3b400640 1 -- 172.21.15.106:0/3141029862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d20003520 con 0x7f5d20004800 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.827+0000 7f5d3be00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d3b400640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d20004800 0x7f5d20004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800ba30 tx=0x7f5d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4b2f36dcfab31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28030070 con 0x7f5d20004800 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2802ae10 con 0x7f5d20004800 2024-04-26T21:46:33.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d3aa00640 1 -- 172.21.15.106:0/3141029862 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800b600 con 0x7f5d20004800 2024-04-26T21:46:33.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d41c00640 1 -- 172.21.15.106:0/3141029862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d20004800 msgr2=0x7f5d20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.828+0000 7f5d41c00640 1 --2- 172.21.15.106:0/3141029862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d20004800 0x7f5d20004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800ba30 tx=0x7f5d20005830 comp rx=0 tx=0).stop 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.829+0000 7f5d41200640 1 -- 172.21.15.106:0/3141029862 reap_dead start 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.829+0000 7f5d41c00640 1 -- 172.21.15.106:0/3141029862 shutdown_connections 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.829+0000 7f5d41c00640 1 -- 172.21.15.106:0/3141029862 >> 172.21.15.106:0/3141029862 conn(0x7f5d3c09e2f0 msgr2=0x7f5d3c0d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.829+0000 7f5d41c00640 1 -- 172.21.15.106:0/3141029862 shutdown_connections 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.829+0000 7f5d41c00640 1 -- 172.21.15.106:0/3141029862 wait complete. 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.830+0000 7f5d41c00640 1 Processor -- start 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 -- start start 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d20004800 0x7f5d3c0ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c0d0a50 0x7f5d3c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c163d80 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c163c00 con 0x7f5d3c0d0a50 2024-04-26T21:46:33.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.831+0000 7f5d41c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c163f00 con 0x7f5d20004800 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46944/0 (socket says 172.21.15.106:46944) 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 -- 172.21.15.106:0/1216873595 learned_addr learned my addr 172.21.15.106:0/1216873595 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 -- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d20004800 msgr2=0x7f5d3c0ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d20004800 0x7f5d3c0ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 -- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c0d0a50 msgr2=0x7f5d3c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d41200640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c0d0a50 0x7f5d3c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c0d0a50 0x7f5d3c0b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 -- 172.21.15.106:0/1216873595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c0c8e90 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.832+0000 7f5d3b400640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800bb50 tx=0x7f5d28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.833+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28043030 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.833+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2802ac10 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.833+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800b600 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.833+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c0c9170 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.833+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c0c9730 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.834+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c0693c0 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.835+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d28046070 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.836+0000 7f5d33e00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 0x7f5d1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.836+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d280ac570 con 0x7f5d3c0ce9a0 2024-04-26T21:46:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.837+0000 7f5d3be00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 0x7f5d1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.837+0000 7f5d3be00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 0x7f5d1c06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007460 tx=0x7f5d2c009750 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:33.841+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d280333b0 con 0x7f5d3c0ce9a0 2024-04-26T21:46:34.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.043+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5d3c0d0e30 con 0x7f5d3c0ce9a0 2024-04-26T21:46:34.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.044+0000 7f5d33e00640 1 -- 172.21.15.106:0/1216873595 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5d280aae20 con 0x7f5d3c0ce9a0 2024-04-26T21:46:34.045 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.046+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 msgr2=0x7f5d1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.046+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 0x7f5d1c06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007460 tx=0x7f5d2c009750 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.046+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 msgr2=0x7f5d3c0d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.046+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800bb50 tx=0x7f5d28004730 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 shutdown_connections 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c0d0a50 0x7f5d3c0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c0ce9a0 0x7f5d3c0d0510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5d1c06c920 0x7f5d1c06ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 --2- 172.21.15.106:0/1216873595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d20004800 0x7f5d3c0ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 >> 172.21.15.106:0/1216873595 conn(0x7f5d3c09e2f0 msgr2=0x7f5d3c161960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 shutdown_connections 2024-04-26T21:46:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.047+0000 7f5d41c00640 1 -- 172.21.15.106:0/1216873595 wait complete. 2024-04-26T21:46:34.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:34.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.267+0000 7f2411800640 1 Processor -- start 2024-04-26T21:46:34.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.267+0000 7f2411800640 1 -- start start 2024-04-26T21:46:34.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.268+0000 7f2411800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f240c06c770 0x7f240c06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.268+0000 7f2411800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f240c15cb70 0x7f240c073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.268+0000 7f2411800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c06d090 con 0x7f240c15cfd0 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.268+0000 7f2411800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f240c0af460 con 0x7f240c15cb70 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.268+0000 7f2411800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f240c0ae850 con 0x7f240c06c770 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f240c15cb70 0x7f240c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f240c15cb70 0x7f240c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46956/0 (socket says 172.21.15.106:46956) 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 -- 172.21.15.106:0/3645652676 learned_addr learned my addr 172.21.15.106:0/3645652676 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 -- 172.21.15.106:0/3645652676 >> v2:172.21.15.204:3300/0 conn(0x7f240c06c770 msgr2=0x7f240c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:34.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 --2- 172.21.15.106:0/3645652676 >> v2:172.21.15.204:3300/0 conn(0x7f240c06c770 0x7f240c06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 -- 172.21.15.106:0/3645652676 >> v1:172.21.15.204:6789/0 conn(0x7f240c15cfd0 legacy=0x7f240c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:34.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.269+0000 7f240a000640 1 -- 172.21.15.106:0/3645652676 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c0aeb30 con 0x7f240c15cb70 2024-04-26T21:46:34.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.270+0000 7f240a000640 1 --2- 172.21.15.106:0/3645652676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cb70 0x7f240c073120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f240c0741f0 tx=0x7f23f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302cc782570a27f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.270+0000 7f2409600640 1 -- 172.21.15.106:0/3645652676 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8037070 con 0x7f240c15cb70 2024-04-26T21:46:34.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2409600640 1 -- 172.21.15.106:0/3645652676 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f8032440 con 0x7f240c15cb70 2024-04-26T21:46:34.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2409600640 1 -- 172.21.15.106:0/3645652676 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8032810 con 0x7f240c15cb70 2024-04-26T21:46:34.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 -- 172.21.15.106:0/3645652676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cb70 msgr2=0x7f240c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 --2- 172.21.15.106:0/3645652676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cb70 0x7f240c073120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f240c0741f0 tx=0x7f23f802a230 comp rx=0 tx=0).stop 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 -- 172.21.15.106:0/3645652676 shutdown_connections 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 --2- 172.21.15.106:0/3645652676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cb70 0x7f240c073120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 --2- 172.21.15.106:0/3645652676 >> v2:172.21.15.204:3300/0 conn(0x7f240c06c770 0x7f240c06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 -- 172.21.15.106:0/3645652676 >> 172.21.15.106:0/3645652676 conn(0x7f240c0b2d50 msgr2=0x7f240c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 -- 172.21.15.106:0/3645652676 shutdown_connections 2024-04-26T21:46:34.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.271+0000 7f2411800640 1 -- 172.21.15.106:0/3645652676 wait complete. 2024-04-26T21:46:34.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.273+0000 7f2411800640 1 Processor -- start 2024-04-26T21:46:34.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.273+0000 7f2411800640 1 -- start start 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f240c06c770 0x7f240c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f240c15cb70 0x7f240c14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f240c15c600 con 0x7f240c15cfd0 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f240c0af0f0 con 0x7f240c15cb70 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f2411800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f240c15c780 con 0x7f240c06c770 2024-04-26T21:46:34.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.274+0000 7f240b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46960/0 (socket says 172.21.15.106:46960) 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 -- 172.21.15.106:0/1226058174 learned_addr learned my addr 172.21.15.106:0/1226058174 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240a000640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f240c15cb70 0x7f240c14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240aa00640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f240c06c770 0x7f240c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 -- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f240c06c770 msgr2=0x7f240c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f240c06c770 0x7f240c0cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 -- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f240c15cb70 msgr2=0x7f240c14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f240c15cb70 0x7f240c14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 -- 172.21.15.106:0/1226058174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c0cad40 con 0x7f240c15cfd0 2024-04-26T21:46:34.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.275+0000 7f240b400640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f240400b990 tx=0x7f240400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.276+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404019070 con 0x7f240c15cfd0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.276+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240400cba0 con 0x7f240c15cfd0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.276+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404014440 con 0x7f240c15cfd0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.276+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c0cb020 con 0x7f240c15cfd0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.276+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c0cb5e0 con 0x7f240c15cfd0 2024-04-26T21:46:34.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.277+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c069540 con 0x7f240c15cfd0 2024-04-26T21:46:34.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.278+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f240401c070 con 0x7f240c15cfd0 2024-04-26T21:46:34.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.279+0000 7f2402a00640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.279+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f240408b970 con 0x7f240c15cfd0 2024-04-26T21:46:34.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.279+0000 7f240aa00640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.280+0000 7f240aa00640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009890 tx=0x7f23fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.283+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2404059d00 con 0x7f240c15cfd0 2024-04-26T21:46:34.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.479+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f240c0b33d0 con 0x7f240c15cfd0 2024-04-26T21:46:34.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.481+0000 7f2402a00640 1 -- 172.21.15.106:0/1226058174 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f2404059600 con 0x7f240c15cfd0 2024-04-26T21:46:34.482 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:34.482 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:34.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.483+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 msgr2=0x7f23ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009890 tx=0x7f23fc008040 comp rx=0 tx=0).stop 2024-04-26T21:46:34.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 msgr2=0x7f240c15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f240400b990 tx=0x7f240400be50 comp rx=0 tx=0).stop 2024-04-26T21:46:34.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 shutdown_connections 2024-04-26T21:46:34.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f240c15cfd0 0x7f240c15e6f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f240c15cb70 0x7f240c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 --2- 172.21.15.106:0/1226058174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f240c06c770 0x7f240c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 >> 172.21.15.106:0/1226058174 conn(0x7f240c0b2d50 msgr2=0x7f240c15dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 shutdown_connections 2024-04-26T21:46:34.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.484+0000 7f2411800640 1 -- 172.21.15.106:0/1226058174 wait complete. 2024-04-26T21:46:34.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:34.504 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:34.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.700+0000 7f4e89600640 1 Processor -- start 2024-04-26T21:46:34.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.700+0000 7f4e89600640 1 -- start start 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4e84147f20 0x7f4e840b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84073da0 0x7f4e8406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e8406ccb0 0x7f4e84072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8406d100 con 0x7f4e8406ccb0 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4e840ade20 con 0x7f4e84147f20 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e89600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4e840ae100 con 0x7f4e84073da0 2024-04-26T21:46:34.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.702+0000 7f4e83400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e8406ccb0 0x7f4e84072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e8406ccb0 0x7f4e84072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46964/0 (socket says 172.21.15.106:46964) 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 -- 172.21.15.106:0/1615490298 learned_addr learned my addr 172.21.15.106:0/1615490298 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e82a00640 1 --2- 172.21.15.106:0/1615490298 >> v2:172.21.15.204:3300/0 conn(0x7f4e84147f20 0x7f4e840b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e82000640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84073da0 0x7f4e8406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 -- 172.21.15.106:0/1615490298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84073da0 msgr2=0x7f4e8406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84073da0 0x7f4e8406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 -- 172.21.15.106:0/1615490298 >> v2:172.21.15.204:3300/0 conn(0x7f4e84147f20 msgr2=0x7f4e840b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:34.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 --2- 172.21.15.106:0/1615490298 >> v2:172.21.15.204:3300/0 conn(0x7f4e84147f20 0x7f4e840b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.703+0000 7f4e83400640 1 -- 172.21.15.106:0/1615490298 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e840ae3e0 con 0x7f4e8406ccb0 2024-04-26T21:46:34.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.704+0000 7f4e83400640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e8406ccb0 0x7f4e84072010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e84075d20 tx=0x7f4e7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1636c2a27198549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e81600640 1 -- 172.21.15.106:0/1615490298 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c036070 con 0x7f4e8406ccb0 2024-04-26T21:46:34.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e81600640 1 -- 172.21.15.106:0/1615490298 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7c031600 con 0x7f4e8406ccb0 2024-04-26T21:46:34.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e81600640 1 -- 172.21.15.106:0/1615490298 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0319d0 con 0x7f4e8406ccb0 2024-04-26T21:46:34.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 -- 172.21.15.106:0/1615490298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e8406ccb0 msgr2=0x7f4e84072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e8406ccb0 0x7f4e84072010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e84075d20 tx=0x7f4e7c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 -- 172.21.15.106:0/1615490298 shutdown_connections 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e8406ccb0 0x7f4e84072010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 --2- 172.21.15.106:0/1615490298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84073da0 0x7f4e8406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 --2- 172.21.15.106:0/1615490298 >> v2:172.21.15.204:3300/0 conn(0x7f4e84147f20 0x7f4e840b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.705+0000 7f4e89600640 1 -- 172.21.15.106:0/1615490298 >> 172.21.15.106:0/1615490298 conn(0x7f4e840b2d50 msgr2=0x7f4e840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.706+0000 7f4e89600640 1 -- 172.21.15.106:0/1615490298 shutdown_connections 2024-04-26T21:46:34.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.706+0000 7f4e89600640 1 -- 172.21.15.106:0/1615490298 wait complete. 2024-04-26T21:46:34.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.707+0000 7f4e89600640 1 Processor -- start 2024-04-26T21:46:34.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.707+0000 7f4e89600640 1 -- start start 2024-04-26T21:46:34.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e8406ccb0 0x7f4e8415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 0x7f4e840cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84074230 con 0x7f4e84147f20 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e840ae9a0 con 0x7f4e84073da0 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.708+0000 7f4e89600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8406e220 con 0x7f4e8406ccb0 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46970/0 (socket says 172.21.15.106:46970) 2024-04-26T21:46:34.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 -- 172.21.15.106:0/1839157141 learned_addr learned my addr 172.21.15.106:0/1839157141 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:34.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e82a00640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e8406ccb0 0x7f4e8415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 -- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e8406ccb0 msgr2=0x7f4e8415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e82000640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 0x7f4e840cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e8406ccb0 0x7f4e8415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 -- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 msgr2=0x7f4e840cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 0x7f4e840cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e83400640 1 -- 172.21.15.106:0/1839157141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e841434c0 con 0x7f4e84147f20 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.709+0000 7f4e82000640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 0x7f4e840cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e83400640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009f40 tx=0x7f4e7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c036050 con 0x7f4e84147f20 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7c042070 con 0x7f4e84147f20 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c009510 con 0x7f4e84147f20 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e841437a0 con 0x7f4e84147f20 2024-04-26T21:46:34.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.710+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e84143d60 con 0x7f4e84147f20 2024-04-26T21:46:34.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.711+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e84069650 con 0x7f4e84147f20 2024-04-26T21:46:34.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.712+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e7c007bf0 con 0x7f4e84147f20 2024-04-26T21:46:34.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.713+0000 7f4e7aa00640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 0x7f4e6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:34.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.713+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c0ac960 con 0x7f4e84147f20 2024-04-26T21:46:34.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.713+0000 7f4e82a00640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 0x7f4e6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:34.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.714+0000 7f4e82a00640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 0x7f4e6406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e840b04d0 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.717+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e7c07acf0 con 0x7f4e84147f20 2024-04-26T21:46:34.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.917+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e840b5f60 con 0x7f4e84147f20 2024-04-26T21:46:34.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.917+0000 7f4e7aa00640 1 -- 172.21.15.106:0/1839157141 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4e7c07a5f0 con 0x7f4e84147f20 2024-04-26T21:46:34.919 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:34.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 msgr2=0x7f4e6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 0x7f4e6406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e840b04d0 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 msgr2=0x7f4e840cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009f40 tx=0x7f4e7c004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 shutdown_connections 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e84147f20 0x7f4e840cdc10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073da0 0x7f4e840cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4e6406c8f0 0x7f4e6406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 --2- 172.21.15.106:0/1839157141 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e8406ccb0 0x7f4e8415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.920+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 >> 172.21.15.106:0/1839157141 conn(0x7f4e840b2d50 msgr2=0x7f4e840b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:34.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.921+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 shutdown_connections 2024-04-26T21:46:34.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:34.921+0000 7f4e89600640 1 -- 172.21.15.106:0/1839157141 wait complete. 2024-04-26T21:46:34.939 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.128+0000 7f36c6e00640 1 Processor -- start 2024-04-26T21:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.128+0000 7f36c6e00640 1 -- start start 2024-04-26T21:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36c00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36c015fbd0 0x7f36c00d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c00d0170 con 0x7f36c0162cf0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f36c00d0450 con 0x7f36c015fbd0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36c00d0730 con 0x7f36c015f7f0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f36c0162cf0 0x7f36c0163180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43630/0 (socket says 172.21.15.106:43630) 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c6400640 1 -- 172.21.15.106:0/3597221380 learned_addr learned my addr 172.21.15.106:0/3597221380 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587630103 0 0) 0x7f36c00d0170 con 0x7f36c0162cf0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c5a00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36c00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36a4003520 con 0x7f36c0162cf0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c5000640 1 --2- 172.21.15.106:0/3597221380 >> v2:172.21.15.204:3300/0 conn(0x7f36c015fbd0 0x7f36c00d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1346107924 0 0) 0x7f36a4003520 con 0x7f36c0162cf0 2024-04-26T21:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 msgr2=0x7f36c00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36c00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 >> v2:172.21.15.204:3300/0 conn(0x7f36c015fbd0 msgr2=0x7f36c00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 --2- 172.21.15.106:0/3597221380 >> v2:172.21.15.204:3300/0 conn(0x7f36c015fbd0 0x7f36c00d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c00d0a10 con 0x7f36c0162cf0 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f36b80019f0 con 0x7f36c0162cf0 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 >> v1:172.21.15.106:6789/0 conn(0x7f36c0162cf0 legacy=0x7f36c0163180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36a4003a20 0x7f36a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4004800 0x7f36a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36c00d0450 con 0x7f36a4003a20 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36c00d0730 con 0x7f36a4004800 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36a400bb70 con 0x7f36c015f7f0 2024-04-26T21:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.130+0000 7f36c5a00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36a4003a20 0x7f36a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c6400640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5a00640 1 -- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 msgr2=0x7f36a4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5000640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4004800 0x7f36a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5a00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c015f7f0 0x7f36a4005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5a00640 1 -- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4004800 msgr2=0x7f36a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5a00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4004800 0x7f36a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5a00640 1 -- 172.21.15.106:0/3597221380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4003520 con 0x7f36a4003a20 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.131+0000 7f36c5000640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4004800 0x7f36a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36c5a00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36a4003a20 0x7f36a4003e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ba30 tx=0x7f36a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daec52b436c0a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0030070 con 0x7f36a4003a20 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b002ade0 con 0x7f36a4003a20 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36bfe00640 1 -- 172.21.15.106:0/3597221380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b000b600 con 0x7f36a4003a20 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36c6e00640 1 -- 172.21.15.106:0/3597221380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36a4003a20 msgr2=0x7f36a4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36c6e00640 1 --2- 172.21.15.106:0/3597221380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36a4003a20 0x7f36a4003e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ba30 tx=0x7f36a4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.132+0000 7f36c6400640 1 -- 172.21.15.106:0/3597221380 reap_dead start 2024-04-26T21:46:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.133+0000 7f36c6e00640 1 -- 172.21.15.106:0/3597221380 shutdown_connections 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.133+0000 7f36c6e00640 1 -- 172.21.15.106:0/3597221380 >> 172.21.15.106:0/3597221380 conn(0x7f36c00d4f80 msgr2=0x7f36c00d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.133+0000 7f36c6e00640 1 -- 172.21.15.106:0/3597221380 shutdown_connections 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.133+0000 7f36c6e00640 1 -- 172.21.15.106:0/3597221380 wait complete. 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.134+0000 7f36c6e00640 1 Processor -- start 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.134+0000 7f36c6e00640 1 -- start start 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4003a20 0x7f36c015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c00cb070 0x7f36c01f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0160050 con 0x7f36c00cb5c0 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36c00d2810 con 0x7f36a4003a20 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.135+0000 7f36c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36c015f430 con 0x7f36c00cb070 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46986/0 (socket says 172.21.15.106:46986) 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 -- 172.21.15.106:0/1090239280 learned_addr learned my addr 172.21.15.106:0/1090239280 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 -- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c00cb070 msgr2=0x7f36c01f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c00cb070 0x7f36c01f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 -- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4003a20 msgr2=0x7f36c015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4003a20 0x7f36c015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 -- 172.21.15.106:0/1090239280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01f69f0 con 0x7f36c00cb5c0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.136+0000 7f36c6400640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36c00c8990 tx=0x7f36b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.137+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8004d60 con 0x7f36c00cb5c0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.137+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b800bb80 con 0x7f36c00cb5c0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.137+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8007070 con 0x7f36c00cb5c0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.137+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c00c8bc0 con 0x7f36c00cb5c0 2024-04-26T21:46:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.137+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c00c9180 con 0x7f36c00cb5c0 2024-04-26T21:46:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.138+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c00693c0 con 0x7f36c00cb5c0 2024-04-26T21:46:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.139+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36b801b020 con 0x7f36c00cb5c0 2024-04-26T21:46:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.140+0000 7f36bd600640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 0x7f36a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.140+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36b808bc80 con 0x7f36c00cb5c0 2024-04-26T21:46:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.140+0000 7f36c5a00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 0x7f36a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.140+0000 7f36c5a00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 0x7f36a006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ba00 tx=0x7f36b002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.144+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36b805a010 con 0x7f36c00cb5c0 2024-04-26T21:46:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.341+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f36c008e780 con 0x7f36c00cb5c0 2024-04-26T21:46:35.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.342+0000 7f36bd600640 1 -- 172.21.15.106:0/1090239280 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f36b8059910 con 0x7f36c00cb5c0 2024-04-26T21:46:35.343 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 msgr2=0x7f36a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 0x7f36a006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ba00 tx=0x7f36b002c000 comp rx=0 tx=0).stop 2024-04-26T21:46:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 msgr2=0x7f36c0160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36c00c8990 tx=0x7f36b800b4a0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 shutdown_connections 2024-04-26T21:46:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c00cb5c0 0x7f36c0160b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f36a006c920 0x7f36a006ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c00cb070 0x7f36c01f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 --2- 172.21.15.106:0/1090239280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36a4003a20 0x7f36c015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 >> 172.21.15.106:0/1090239280 conn(0x7f36c00d4f80 msgr2=0x7f36c00b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:35.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 shutdown_connections 2024-04-26T21:46:35.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.345+0000 7f36c6e00640 1 -- 172.21.15.106:0/1090239280 wait complete. 2024-04-26T21:46:35.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.566+0000 7f2c19200640 1 Processor -- start 2024-04-26T21:46:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.566+0000 7f2c19200640 1 -- start start 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c19200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2c140d1da0 0x7f2c14162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c19200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c140d3830 con 0x7f2c140d19c0 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c19200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c140d3b10 con 0x7f2c140b6170 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c19200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2c140d40d0 con 0x7f2c140d1da0 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c11e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2c140d19c0 0x7f2c140d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43644/0 (socket says 172.21.15.106:43644) 2024-04-26T21:46:35.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.567+0000 7f2c11e00640 1 -- 172.21.15.106:0/2551820607 learned_addr learned my addr 172.21.15.106:0/2551820607 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:35.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c12800640 1 --2- 172.21.15.106:0/2551820607 >> v2:172.21.15.204:3300/0 conn(0x7f2c140d1da0 0x7f2c14162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397075356 0 0) 0x7f2c140d3830 con 0x7f2c140d19c0 2024-04-26T21:46:35.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf8003520 con 0x7f2c140d19c0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 535365739 0 0) 0x7f2bf8003520 con 0x7f2c140d19c0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 >> v2:172.21.15.204:3300/0 conn(0x7f2c140d1da0 msgr2=0x7f2c14162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 --2- 172.21.15.106:0/2551820607 >> v2:172.21.15.204:3300/0 conn(0x7f2c140d1da0 0x7f2c14162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 >> v1:172.21.15.204:6789/0 conn(0x7f2c140b6170 legacy=0x7f2c140c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c140d3df0 con 0x7f2c140d19c0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2c04001670 con 0x7f2c140d19c0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 >> v1:172.21.15.106:6789/0 conn(0x7f2c140d19c0 legacy=0x7f2c140d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf80039b0 0x7f2bf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 0x7f2bf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf80051c0 0x7f2bf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800c130 con 0x7f2bf8004800 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800c2e0 con 0x7f2bf80051c0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.568+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf800c4e0 con 0x7f2bf80039b0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 0x7f2bf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 -- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf80039b0 msgr2=0x7f2bf8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf80039b0 0x7f2bf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c12800640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf80051c0 0x7f2bf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 -- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf80051c0 msgr2=0x7f2bf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf80051c0 0x7f2bf800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c11400640 1 -- 172.21.15.106:0/2551820607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf8003520 con 0x7f2bf8004800 2024-04-26T21:46:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.569+0000 7f2c12800640 1 -- 172.21.15.106:0/2551820607 reap_dead start 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.570+0000 7f2c11400640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 0x7f2bf8004c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc001750 tx=0x7f2bf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc052abe9e613854 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.570+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc001370 con 0x7f2bf8004800 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.570+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0052d0 con 0x7f2bf8004800 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.570+0000 7f2c10a00640 1 -- 172.21.15.106:0/2551820607 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc02abc0 con 0x7f2bf8004800 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 -- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 msgr2=0x7f2bf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 0x7f2bf8004c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc001750 tx=0x7f2bf8005830 comp rx=0 tx=0).stop 2024-04-26T21:46:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 -- 172.21.15.106:0/2551820607 shutdown_connections 2024-04-26T21:46:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 --2- 172.21.15.106:0/2551820607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf8004800 0x7f2bf8004c80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 -- 172.21.15.106:0/2551820607 >> 172.21.15.106:0/2551820607 conn(0x7f2c1409e460 msgr2=0x7f2c140d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 -- 172.21.15.106:0/2551820607 shutdown_connections 2024-04-26T21:46:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.571+0000 7f2c19200640 1 -- 172.21.15.106:0/2551820607 wait complete. 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.573+0000 7f2c19200640 1 Processor -- start 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.573+0000 7f2c19200640 1 -- start start 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf8004800 0x7f2c140b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c140ba740 0x7f2c140cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140c8df0 con 0x7f2c1415d960 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140c8c70 con 0x7f2c140ba740 2024-04-26T21:46:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c19200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c140c8f70 con 0x7f2bf8004800 2024-04-26T21:46:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c11400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c11400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47012/0 (socket says 172.21.15.106:47012) 2024-04-26T21:46:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.574+0000 7f2c11400640 1 -- 172.21.15.106:0/2915607955 learned_addr learned my addr 172.21.15.106:0/2915607955 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11e00640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf8004800 0x7f2c140b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 -- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf8004800 msgr2=0x7f2c140b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf8004800 0x7f2c140b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 -- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c140ba740 msgr2=0x7f2c140cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c140ba740 0x7f2c140cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 -- 172.21.15.106:0/2915607955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c140cd3e0 con 0x7f2c1415d960 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c11400640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc001870 tx=0x7f2bfc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc03f030 con 0x7f2c1415d960 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc005480 con 0x7f2c1415d960 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc037040 con 0x7f2c1415d960 2024-04-26T21:46:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.575+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c140cd6c0 con 0x7f2c1415d960 2024-04-26T21:46:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.576+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c140babd0 con 0x7f2c1415d960 2024-04-26T21:46:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.577+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c14069650 con 0x7f2c1415d960 2024-04-26T21:46:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.577+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bfc042070 con 0x7f2c1415d960 2024-04-26T21:46:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.578+0000 7f2c0a000640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.578+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc0ab160 con 0x7f2c1415d960 2024-04-26T21:46:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.579+0000 7f2c11e00640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.579+0000 7f2c11e00640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c04001860 tx=0x7f2c04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.583+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bfc079460 con 0x7f2c1415d960 2024-04-26T21:46:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.779+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c140d26d0 con 0x7f2c1415d960 2024-04-26T21:46:35.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.780+0000 7f2c0a000640 1 -- 172.21.15.106:0/2915607955 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2bfc078d60 con 0x7f2c1415d960 2024-04-26T21:46:35.780 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 msgr2=0x7f2bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c04001860 tx=0x7f2c04009040 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 msgr2=0x7f2c140ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc001870 tx=0x7f2bfc001120 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 shutdown_connections 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c140ba740 0x7f2c140cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1415d960 0x7f2c140ba200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2bf006c8f0 0x7f2bf006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 --2- 172.21.15.106:0/2915607955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf8004800 0x7f2c140b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 >> 172.21.15.106:0/2915607955 conn(0x7f2c1409e460 msgr2=0x7f2c140d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 shutdown_connections 2024-04-26T21:46:35.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:35.781+0000 7f2c19200640 1 -- 172.21.15.106:0/2915607955 wait complete. 2024-04-26T21:46:35.863 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-04-26T21:46:35.864 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:46:35.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.064+0000 7f8963e00640 1 Processor -- start 2024-04-26T21:46:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.064+0000 7f8963e00640 1 -- start start 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.065+0000 7f8963e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f89640b5660 0x7f896406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.065+0000 7f8963e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f896406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8963e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89640727f0 con 0x7f89640d0e70 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8963e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f89640b4a10 con 0x7f89640b5660 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8963e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f89640b4cf0 con 0x7f89640b5d40 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8962a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f89640d0e70 0x7f89640d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43652/0 (socket says 172.21.15.106:43652) 2024-04-26T21:46:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8962a00640 1 -- 172.21.15.106:0/3651262794 learned_addr learned my addr 172.21.15.106:0/3651262794 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:36.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f895a000640 1 --2- 172.21.15.106:0/3651262794 >> v2:172.21.15.204:3300/0 conn(0x7f89640b5660 0x7f896406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8963400640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f896406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988540848 0 0) 0x7f89640727f0 con 0x7f89640d0e70 2024-04-26T21:46:36.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.066+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8948003520 con 0x7f89640d0e70 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3659504029 0 0) 0x7f8948003520 con 0x7f89640d0e70 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 msgr2=0x7f896406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f896406e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 >> v2:172.21.15.204:3300/0 conn(0x7f89640b5660 msgr2=0x7f896406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 --2- 172.21.15.106:0/3651262794 >> v2:172.21.15.204:3300/0 conn(0x7f89640b5660 0x7f896406e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89640b4fd0 con 0x7f89640d0e70 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f8954001670 con 0x7f89640d0e70 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 >> v1:172.21.15.106:6789/0 conn(0x7f89640d0e70 legacy=0x7f89640d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89480039b0 0x7f8948003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8948004800 0x7f8948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f89480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89640b4a10 con 0x7f89480039b0 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89640b4cf0 con 0x7f8948004800 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f894800b9c0 con 0x7f89640b5d40 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.067+0000 7f8962a00640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89480039b0 0x7f8948003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8962a00640 1 -- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 msgr2=0x7f89480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8963400640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f89480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8962a00640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89640b5d40 0x7f89480051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8962a00640 1 -- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8948004800 msgr2=0x7f8948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8962a00640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8948004800 0x7f8948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.068+0000 7f8962a00640 1 -- 172.21.15.106:0/3651262794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8948003520 con 0x7f89480039b0 2024-04-26T21:46:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8962a00640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89480039b0 0x7f8948003dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f895400c2c0 tx=0x7f8948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb7872f0d3bd752 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954007d50 con 0x7f89480039b0 2024-04-26T21:46:36.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895402e070 con 0x7f89480039b0 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8962000640 1 -- 172.21.15.106:0/3651262794 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954005d30 con 0x7f89480039b0 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8963e00640 1 -- 172.21.15.106:0/3651262794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89480039b0 msgr2=0x7f8948003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8963e00640 1 --2- 172.21.15.106:0/3651262794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89480039b0 0x7f8948003dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f895400c2c0 tx=0x7f8948005830 comp rx=0 tx=0).stop 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8963400640 1 -- 172.21.15.106:0/3651262794 reap_dead start 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8963e00640 1 -- 172.21.15.106:0/3651262794 shutdown_connections 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.069+0000 7f8963e00640 1 -- 172.21.15.106:0/3651262794 >> 172.21.15.106:0/3651262794 conn(0x7f896406c740 msgr2=0x7f896406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.070+0000 7f8963e00640 1 -- 172.21.15.106:0/3651262794 shutdown_connections 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.070+0000 7f8963e00640 1 -- 172.21.15.106:0/3651262794 wait complete. 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.071+0000 7f8963e00640 1 Processor -- start 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.071+0000 7f8963e00640 1 -- start start 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89480039b0 0x7f8964160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8964160fe0 0x7f89640b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f896415f320 con 0x7f8964148530 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f896415f1a0 con 0x7f89480039b0 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.072+0000 7f8963e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f896406e930 con 0x7f8964160fe0 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47026/0 (socket says 172.21.15.106:47026) 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 -- 172.21.15.106:0/3317921085 learned_addr learned my addr 172.21.15.106:0/3317921085 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 -- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8964160fe0 msgr2=0x7f89640b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8964160fe0 0x7f89640b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 -- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89480039b0 msgr2=0x7f8964160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f8962a00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89480039b0 0x7f8964160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89480039b0 0x7f8964160400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 -- 172.21.15.106:0/3317921085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89641528a0 con 0x7f8964148530 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.073+0000 7f895a000640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e7d0 tx=0x7f894c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.074+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017070 con 0x7f8964148530 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.074+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c00c860 con 0x7f8964148530 2024-04-26T21:46:36.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.074+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8964161610 con 0x7f8964148530 2024-04-26T21:46:36.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.074+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c00cc30 con 0x7f8964148530 2024-04-26T21:46:36.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.074+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896415f920 con 0x7f8964148530 2024-04-26T21:46:36.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.075+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89640693c0 con 0x7f8964148530 2024-04-26T21:46:36.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.076+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894c01a070 con 0x7f8964148530 2024-04-26T21:46:36.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.077+0000 7f895b400640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.077+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f894c08b920 con 0x7f8964148530 2024-04-26T21:46:36.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.077+0000 7f8962a00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.078+0000 7f8962a00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8954033000 tx=0x7f8954009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.081+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f894c059cb0 con 0x7f8964148530 2024-04-26T21:46:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.268+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89640afd40 con 0x7f8964148530 2024-04-26T21:46:36.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.271+0000 7f895b400640 1 -- 172.21.15.106:0/3317921085 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16617 (secure 0 0 0) 0x7f894c0595b0 con 0x7f8964148530 2024-04-26T21:46:36.272 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:36.272 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:45:57.680261+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 msgr2=0x7f894006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8954033000 tx=0x7f8954009030 comp rx=0 tx=0).stop 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 msgr2=0x7f8964160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e7d0 tx=0x7f894c00ec90 comp rx=0 tx=0).stop 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 shutdown_connections 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8964160fe0 0x7f89640b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8964148530 0x7f8964160aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 --2- 172.21.15.106:0/3317921085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89480039b0 0x7f8964160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 >> 172.21.15.106:0/3317921085 conn(0x7f896406c740 msgr2=0x7f896406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:36.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 shutdown_connections 2024-04-26T21:46:36.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.274+0000 7f8963e00640 1 -- 172.21.15.106:0/3317921085 wait complete. 2024-04-26T21:46:36.294 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:36.294 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:36.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.496+0000 7f2816400640 1 Processor -- start 2024-04-26T21:46:36.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.496+0000 7f2816400640 1 -- start start 2024-04-26T21:46:36.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f2816400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f2816400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28100b5ca0 0x7f281006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f2816400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28100af800 con 0x7f28100add80 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f2816400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f28100ae340 con 0x7f28100b5ca0 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f2816400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28100afaa0 con 0x7f281015d100 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f280f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f280ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28100b5ca0 0x7f281006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.498+0000 7f280f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:45530/0 (socket says 172.21.15.106:45530) 2024-04-26T21:46:36.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280f400640 1 -- 172.21.15.106:0/3910134151 learned_addr learned my addr 172.21.15.106:0/3910134151 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:36.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280ea00640 1 -- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 msgr2=0x7f281006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280ea00640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280ea00640 1 -- 172.21.15.106:0/3910134151 >> v1:172.21.15.204:6789/0 conn(0x7f28100add80 legacy=0x7f2810072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280ea00640 1 -- 172.21.15.106:0/3910134151 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28100afd80 con 0x7f28100b5ca0 2024-04-26T21:46:36.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.499+0000 7f280f400640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:36.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.501+0000 7f280ea00640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100b5ca0 0x7f281006ccb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b6f44a75e4becb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.501+0000 7f280e000640 1 -- 172.21.15.106:0/3910134151 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800031600 con 0x7f28100b5ca0 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.501+0000 7f280e000640 1 -- 172.21.15.106:0/3910134151 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28000317a0 con 0x7f28100b5ca0 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.501+0000 7f280e000640 1 -- 172.21.15.106:0/3910134151 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800031b70 con 0x7f28100b5ca0 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 -- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100b5ca0 msgr2=0x7f281006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100b5ca0 0x7f281006ccb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 -- 172.21.15.106:0/3910134151 shutdown_connections 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100b5ca0 0x7f281006ccb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 --2- 172.21.15.106:0/3910134151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 -- 172.21.15.106:0/3910134151 >> 172.21.15.106:0/3910134151 conn(0x7f28100b2d50 msgr2=0x7f28100b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 -- 172.21.15.106:0/3910134151 shutdown_connections 2024-04-26T21:46:36.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.502+0000 7f2816400640 1 -- 172.21.15.106:0/3910134151 wait complete. 2024-04-26T21:46:36.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.504+0000 7f2816400640 1 Processor -- start 2024-04-26T21:46:36.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.504+0000 7f2816400640 1 -- start start 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100add80 0x7f2810156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281009dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f281009ee50 con 0x7f28100b5ca0 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28100b0340 con 0x7f28100add80 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f2816400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2810160b70 con 0x7f281015d100 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f280ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f280ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47028/0 (socket says 172.21.15.106:47028) 2024-04-26T21:46:36.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.505+0000 7f280ea00640 1 -- 172.21.15.106:0/1509510171 learned_addr learned my addr 172.21.15.106:0/1509510171 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280f400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100add80 0x7f2810156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 -- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 msgr2=0x7f281009dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281009dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 -- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100add80 msgr2=0x7f2810156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100add80 0x7f2810156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 -- 172.21.15.106:0/1509510171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28101f5710 con 0x7f28100b5ca0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f280ea00640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2800009eb0 tx=0x7f2800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800040070 con 0x7f28100b5ca0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2800030be0 con 0x7f28100b5ca0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800009450 con 0x7f28100b5ca0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28101f59f0 con 0x7f28100b5ca0 2024-04-26T21:46:36.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.506+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28101f5fb0 con 0x7f28100b5ca0 2024-04-26T21:46:36.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.508+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2810069650 con 0x7f28100b5ca0 2024-04-26T21:46:36.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.508+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2800039040 con 0x7f28100b5ca0 2024-04-26T21:46:36.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.509+0000 7f2807400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.509+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28000aca00 con 0x7f28100b5ca0 2024-04-26T21:46:36.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.510+0000 7f280f400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.510+0000 7f280f400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009a90 tx=0x7f27f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.514+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f280007ae20 con 0x7f28100b5ca0 2024-04-26T21:46:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.720+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28100b1d60 con 0x7f28100b5ca0 2024-04-26T21:46:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.720+0000 7f2807400640 1 -- 172.21.15.106:0/1509510171 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f280007a720 con 0x7f28100b5ca0 2024-04-26T21:46:36.721 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 msgr2=0x7f27ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009a90 tx=0x7f27f8008040 comp rx=0 tx=0).stop 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 msgr2=0x7f281009da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2800009eb0 tx=0x7f2800004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 shutdown_connections 2024-04-26T21:46:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f281015d100 0x7f281009dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28100b5ca0 0x7f281009da60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 --2- 172.21.15.106:0/1509510171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28100add80 0x7f2810156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 >> 172.21.15.106:0/1509510171 conn(0x7f28100b2d50 msgr2=0x7f28100b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 shutdown_connections 2024-04-26T21:46:36.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.723+0000 7f2816400640 1 -- 172.21.15.106:0/1509510171 wait complete. 2024-04-26T21:46:36.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.937+0000 7ff89c200640 1 Processor -- start 2024-04-26T21:46:36.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.937+0000 7ff89c200640 1 -- start start 2024-04-26T21:46:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89c200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff89406ce80 0x7ff8940b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89c200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8940ade20 con 0x7ff894073da0 2024-04-26T21:46:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89c200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8940ae100 con 0x7ff894147f20 2024-04-26T21:46:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89c200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940ae3e0 con 0x7ff89406ce80 2024-04-26T21:46:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89a200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff894073da0 0x7ff8940aeec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43662/0 (socket says 172.21.15.106:43662) 2024-04-26T21:46:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.939+0000 7ff89a200640 1 -- 172.21.15.106:0/1377309403 learned_addr learned my addr 172.21.15.106:0/1377309403 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:36.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff898e00640 1 --2- 172.21.15.106:0/1377309403 >> v2:172.21.15.204:3300/0 conn(0x7ff89406ce80 0x7ff8940b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972242357 0 0) 0x7ff8940ade20 con 0x7ff894073da0 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff87c003520 con 0x7ff894073da0 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3243036581 0 0) 0x7ff87c003520 con 0x7ff894073da0 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 >> v2:172.21.15.204:3300/0 conn(0x7ff89406ce80 msgr2=0x7ff8940b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 --2- 172.21.15.106:0/1377309403 >> v2:172.21.15.204:3300/0 conn(0x7ff89406ce80 0x7ff8940b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 >> v1:172.21.15.204:6789/0 conn(0x7ff894147f20 legacy=0x7ff89406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8940ae6c0 con 0x7ff894073da0 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7ff88c0019f0 con 0x7ff894073da0 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.940+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 >> v1:172.21.15.106:6789/0 conn(0x7ff894073da0 legacy=0x7ff8940aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 0x7ff87c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c004800 0x7ff87c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff87c0051c0 0x7ff87c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940ae3e0 con 0x7ff87c003a00 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c00c0c0 con 0x7ff87c004800 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c00c2c0 con 0x7ff87c0051c0 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 0x7ff87c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff898e00640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c004800 0x7ff87c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 -- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff87c0051c0 msgr2=0x7ff87c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff87c0051c0 0x7ff87c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 -- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c004800 msgr2=0x7ff87c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c004800 0x7ff87c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff899800640 1 -- 172.21.15.106:0/1377309403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c003520 con 0x7ff87c003a00 2024-04-26T21:46:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.941+0000 7ff89a200640 1 -- 172.21.15.106:0/1377309403 reap_dead start 2024-04-26T21:46:36.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.942+0000 7ff899800640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 0x7ff87c003e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff888001750 tx=0x7ff87c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b65c35d1e64c64a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888001370 con 0x7ff87c003a00 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8880052d0 con 0x7ff87c003a00 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff893e00640 1 -- 172.21.15.106:0/1377309403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88802abe0 con 0x7ff87c003a00 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 -- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 msgr2=0x7ff87c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 0x7ff87c003e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff888001750 tx=0x7ff87c005830 comp rx=0 tx=0).stop 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 -- 172.21.15.106:0/1377309403 shutdown_connections 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 --2- 172.21.15.106:0/1377309403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87c003a00 0x7ff87c003e20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 -- 172.21.15.106:0/1377309403 >> 172.21.15.106:0/1377309403 conn(0x7ff8940b2d50 msgr2=0x7ff8940b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 -- 172.21.15.106:0/1377309403 shutdown_connections 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.943+0000 7ff89c200640 1 -- 172.21.15.106:0/1377309403 wait complete. 2024-04-26T21:46:36.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.945+0000 7ff89c200640 1 Processor -- start 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.945+0000 7ff89c200640 1 -- start start 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 0x7ff89415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff89409ba50 0x7ff89409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff89406e220 con 0x7ff89409c410 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940741d0 con 0x7ff87c003a00 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff8940b5c20 con 0x7ff89409ba50 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89a200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89a200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47050/0 (socket says 172.21.15.106:47050) 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.946+0000 7ff89a200640 1 -- 172.21.15.106:0/2991862796 learned_addr learned my addr 172.21.15.106:0/2991862796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff899800640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 0x7ff89415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff898e00640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff89409ba50 0x7ff89409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 -- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff89409ba50 msgr2=0x7ff89409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff89409ba50 0x7ff89409bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 -- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 msgr2=0x7ff89415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 0x7ff89415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 -- 172.21.15.106:0/2991862796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89409c890 con 0x7ff89409c410 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff899800640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 0x7ff89415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:36.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89a200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff89409f0d0 tx=0x7ff88c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c004d60 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c00a5b0 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00a9a0 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.947+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff894161aa0 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.948+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff894162060 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.949+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff894069650 con 0x7ff89409c410 2024-04-26T21:46:36.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.950+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff88c011070 con 0x7ff89409c410 2024-04-26T21:46:36.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.951+0000 7ff891600640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:36.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.951+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff88c08afc0 con 0x7ff89409c410 2024-04-26T21:46:36.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.951+0000 7ff899800640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:36.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.952+0000 7ff899800640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff888032000 tx=0x7ff88802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:36.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:36.955+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff88c059480 con 0x7ff89409c410 2024-04-26T21:46:37.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.160+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff89406cac0 con 0x7ff89409c410 2024-04-26T21:46:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.161+0000 7ff891600640 1 -- 172.21.15.106:0/2991862796 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff88c058d80 con 0x7ff89409c410 2024-04-26T21:46:37.161 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.163+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 msgr2=0x7ff87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.163+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff888032000 tx=0x7ff88802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.163+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 msgr2=0x7ff89409d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.163+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff89409f0d0 tx=0x7ff88c00cbe0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 shutdown_connections 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89409c410 0x7ff89409d850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff89409ba50 0x7ff89409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 --2- 172.21.15.106:0/2991862796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87c003a00 0x7ff89415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 >> 172.21.15.106:0/2991862796 conn(0x7ff8940b2d50 msgr2=0x7ff894146fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 shutdown_connections 2024-04-26T21:46:37.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.164+0000 7ff89c200640 1 -- 172.21.15.106:0/2991862796 wait complete. 2024-04-26T21:46:37.182 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:37.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.384+0000 7fc0cd200640 1 Processor -- start 2024-04-26T21:46:37.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.384+0000 7fc0cd200640 1 -- start start 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0cd200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0c806c770 0x7fc0c806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0cd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 0x7fc0c80b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0cd200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c806eea0 con 0x7fc0c8156650 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0cd200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c806f140 con 0x7fc0c806c770 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0cd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c806f420 con 0x7fc0c80b3c30 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0c5e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc0c8156650 0x7fc0c815cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43672/0 (socket says 172.21.15.106:43672) 2024-04-26T21:46:37.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0c5e00640 1 -- 172.21.15.106:0/589386116 learned_addr learned my addr 172.21.15.106:0/589386116 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0c5400640 1 --2- 172.21.15.106:0/589386116 >> v2:172.21.15.204:3300/0 conn(0x7fc0c806c770 0x7fc0c806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.386+0000 7fc0c6800640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 0x7fc0c80b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106680271 0 0) 0x7fc0c806eea0 con 0x7fc0c8156650 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0ac003520 con 0x7fc0c8156650 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3056090146 0 0) 0x7fc0ac003520 con 0x7fc0c8156650 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 msgr2=0x7fc0c80b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 0x7fc0c80b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 >> v2:172.21.15.204:3300/0 conn(0x7fc0c806c770 msgr2=0x7fc0c806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 --2- 172.21.15.106:0/589386116 >> v2:172.21.15.204:3300/0 conn(0x7fc0c806c770 0x7fc0c806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c806f700 con 0x7fc0c8156650 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fc0b8001670 con 0x7fc0c8156650 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 >> v1:172.21.15.106:6789/0 conn(0x7fc0c8156650 legacy=0x7fc0c815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0ac0039b0 0x7fc0ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0ac004800 0x7fc0ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 0x7fc0ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c806f140 con 0x7fc0ac004800 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c806f420 con 0x7fc0ac0039b0 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.387+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac00dbd0 con 0x7fc0c80b3c30 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0ac004800 0x7fc0ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 -- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 msgr2=0x7fc0ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0c80b3c30 0x7fc0ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 -- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0ac0039b0 msgr2=0x7fc0ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0ac0039b0 0x7fc0ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.388+0000 7fc0c5400640 1 -- 172.21.15.106:0/589386116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac003520 con 0x7fc0ac004800 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.389+0000 7fc0c5400640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0ac004800 0x7fc0ac004c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009ac0 tx=0x7fc0ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31fdd92f59afa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.389+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0036070 con 0x7fc0ac004800 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.389+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b0031400 con 0x7fc0ac004800 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0c4a00640 1 -- 172.21.15.106:0/589386116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0009690 con 0x7fc0ac004800 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 -- 172.21.15.106:0/589386116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0ac004800 msgr2=0x7fc0ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 --2- 172.21.15.106:0/589386116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0ac004800 0x7fc0ac004c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009ac0 tx=0x7fc0ac005830 comp rx=0 tx=0).stop 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0c6800640 1 -- 172.21.15.106:0/589386116 reap_dead start 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 -- 172.21.15.106:0/589386116 shutdown_connections 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 -- 172.21.15.106:0/589386116 >> 172.21.15.106:0/589386116 conn(0x7fc0c80b2d50 msgr2=0x7fc0c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:37.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 -- 172.21.15.106:0/589386116 shutdown_connections 2024-04-26T21:46:37.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.390+0000 7fc0cd200640 1 -- 172.21.15.106:0/589386116 wait complete. 2024-04-26T21:46:37.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.392+0000 7fc0cd200640 1 Processor -- start 2024-04-26T21:46:37.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.392+0000 7fc0cd200640 1 -- start start 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0ac004800 0x7fc0c814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 0x7fc0c80afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c815c680 con 0x7fc0c814d8f0 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c806fcc0 con 0x7fc0c814d490 2024-04-26T21:46:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0cd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c815b100 con 0x7fc0ac004800 2024-04-26T21:46:37.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47074/0 (socket says 172.21.15.106:47074) 2024-04-26T21:46:37.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.393+0000 7fc0c6800640 1 -- 172.21.15.106:0/2694211597 learned_addr learned my addr 172.21.15.106:0/2694211597 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:37.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c5400640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 0x7fc0c80afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c5e00640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0ac004800 0x7fc0c814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 -- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0ac004800 msgr2=0x7fc0c814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0ac004800 0x7fc0c814f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 -- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 msgr2=0x7fc0c80afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 0x7fc0c80afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 -- 172.21.15.106:0/2694211597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80b05d0 con 0x7fc0c814d8f0 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c5400640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 0x7fc0c80afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0c6800640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80d1cf0 tx=0x7fc0c000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000bae0 con 0x7fc0c814d8f0 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c001a070 con 0x7fc0c814d8f0 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.394+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0014400 con 0x7fc0c814d8f0 2024-04-26T21:46:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.395+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c809e730 con 0x7fc0c814d8f0 2024-04-26T21:46:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.395+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c815b630 con 0x7fc0c814d8f0 2024-04-26T21:46:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.396+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8069650 con 0x7fc0c814d8f0 2024-04-26T21:46:37.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.397+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0c0005d10 con 0x7fc0c814d8f0 2024-04-26T21:46:37.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.398+0000 7fc0be000640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 0x7fc0a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.398+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0c008bbc0 con 0x7fc0c814d8f0 2024-04-26T21:46:37.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.398+0000 7fc0c5e00640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 0x7fc0a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.399+0000 7fc0c5e00640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 0x7fc0a406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8001050 tx=0x7fc0b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.403+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0c80b05d0 con 0x7fc0c814d8f0 2024-04-26T21:46:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.597+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0c815cbc0 con 0x7fc0c814d8f0 2024-04-26T21:46:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.598+0000 7fc0be000640 1 -- 172.21.15.106:0/2694211597 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc0c0006000 con 0x7fc0c814d8f0 2024-04-26T21:46:37.598 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:37.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 msgr2=0x7fc0a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 0x7fc0a406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8001050 tx=0x7fc0b8009040 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 msgr2=0x7fc0c80b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80d1cf0 tx=0x7fc0c000b480 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 shutdown_connections 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c814d8f0 0x7fc0c80b0090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c814d490 0x7fc0c80afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc0a406c8f0 0x7fc0a406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 --2- 172.21.15.106:0/2694211597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0ac004800 0x7fc0c814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 >> 172.21.15.106:0/2694211597 conn(0x7fc0c80b2d50 msgr2=0x7fc0c815f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 shutdown_connections 2024-04-26T21:46:37.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.601+0000 7fc0cd200640 1 -- 172.21.15.106:0/2694211597 wait complete. 2024-04-26T21:46:37.620 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-26T21:46:37.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.809+0000 7f2eabe00640 1 Processor -- start 2024-04-26T21:46:37.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.809+0000 7f2eabe00640 1 -- start start 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.811+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 0x7f2eac0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.811+0000 7f2eabe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2eac0cd010 0x7f2eac0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.811+0000 7f2eabe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eac06dee0 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.811+0000 7f2eabe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2eac06f940 con 0x7f2eac0cd010 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.811+0000 7f2eabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac0d0370 con 0x7f2eac1668f0 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eab400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2eac0cbbb0 0x7f2eac166f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43674/0 (socket says 172.21.15.106:43674) 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eab400640 1 -- 172.21.15.106:0/3902857987 learned_addr learned my addr 172.21.15.106:0/3902857987 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 0x7f2eac0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2ea3e00640 1 --2- 172.21.15.106:0/3902857987 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0cd010 0x7f2eac0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 937007865 0 0) 0x7f2eac06dee0 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e90003520 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2449592574 0 0) 0x7f2e90003520 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 msgr2=0x7f2eac0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 0x7f2eac0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0cd010 msgr2=0x7f2eac0cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 --2- 172.21.15.106:0/3902857987 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0cd010 0x7f2eac0cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac06e180 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2ea40019f0 con 0x7f2eac0cbbb0 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.812+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 >> v1:172.21.15.106:6789/0 conn(0x7f2eac0cbbb0 legacy=0x7f2eac166f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e900039b0 0x7f2e90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e90004800 0x7f2e90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 0x7f2e900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac0d0370 con 0x7f2e900039b0 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000b9c0 con 0x7f2e90004800 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9000bb40 con 0x7f2eac1668f0 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e900039b0 0x7f2e90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 -- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 msgr2=0x7f2e900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac1668f0 0x7f2e900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 -- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e90004800 msgr2=0x7f2e90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e90004800 0x7f2e90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.813+0000 7f2eaaa00640 1 -- 172.21.15.106:0/3902857987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e90003520 con 0x7f2e900039b0 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.814+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e900039b0 0x7f2e90003dd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009af0 tx=0x7f2e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5cba1fe60efba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.814+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c036070 con 0x7f2e900039b0 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.814+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c031400 con 0x7f2e900039b0 2024-04-26T21:46:37.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.814+0000 7f2eaa000640 1 -- 172.21.15.106:0/3902857987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c009690 con 0x7f2e900039b0 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 -- 172.21.15.106:0/3902857987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e900039b0 msgr2=0x7f2e90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 --2- 172.21.15.106:0/3902857987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e900039b0 0x7f2e90003dd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009af0 tx=0x7f2e90005830 comp rx=0 tx=0).stop 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eab400640 1 -- 172.21.15.106:0/3902857987 reap_dead start 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 -- 172.21.15.106:0/3902857987 shutdown_connections 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 -- 172.21.15.106:0/3902857987 >> 172.21.15.106:0/3902857987 conn(0x7f2eac0d0ae0 msgr2=0x7f2eac0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 -- 172.21.15.106:0/3902857987 shutdown_connections 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.815+0000 7f2eabe00640 1 -- 172.21.15.106:0/3902857987 wait complete. 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.817+0000 7f2eabe00640 1 Processor -- start 2024-04-26T21:46:37.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.817+0000 7f2eabe00640 1 -- start start 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e900039b0 0x7f2eac15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac160890 0x7f2eac160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac166cd0 con 0x7f2eac0b38a0 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac06fe50 con 0x7f2e900039b0 2024-04-26T21:46:37.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac070130 con 0x7f2eac160890 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57738/0 (socket says 172.21.15.106:57738) 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 -- 172.21.15.106:0/509776453 learned_addr learned my addr 172.21.15.106:0/509776453 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 -- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac160890 msgr2=0x7f2eac160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac160890 0x7f2eac160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 -- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e900039b0 msgr2=0x7f2eac15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e900039b0 0x7f2eac15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.818+0000 7f2eab400640 1 -- 172.21.15.106:0/509776453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac0b45f0 con 0x7f2eac0b38a0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2eab400640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2eac070390 tx=0x7f2ea400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400a7b0 con 0x7f2eac0b38a0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea400a950 con 0x7f2eac0b38a0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400ad00 con 0x7f2eac0b38a0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac0b48d0 con 0x7f2eac0b38a0 2024-04-26T21:46:37.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.819+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac0b4e90 con 0x7f2eac0b38a0 2024-04-26T21:46:37.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.820+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac0693c0 con 0x7f2eac0b38a0 2024-04-26T21:46:37.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.821+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ea4007070 con 0x7f2eac0b38a0 2024-04-26T21:46:37.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.822+0000 7f2ea2a00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 0x7f2e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:37.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.822+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ea408b380 con 0x7f2eac0b38a0 2024-04-26T21:46:37.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.823+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 0x7f2e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:37.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.823+0000 7f2eaaa00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 0x7f2e8806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c033000 tx=0x7f2e9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:37.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:37.826+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ea401a030 con 0x7f2eac0b38a0 2024-04-26T21:46:38.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.109+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f2eac074140 con 0x7f2eac0b38a0 2024-04-26T21:46:38.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.919+0000 7f2ea2a00640 1 -- 172.21.15.106:0/509776453 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v60) ==== 161+0+0 (secure 0 0 0) 0x7f2ea4059710 con 0x7f2eac0b38a0 2024-04-26T21:46:38.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 msgr2=0x7f2e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 0x7f2e8806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c033000 tx=0x7f2e9c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:38.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 msgr2=0x7f2eac0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2eac070390 tx=0x7f2ea400cf00 comp rx=0 tx=0).stop 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 shutdown_connections 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2eac0b38a0 0x7f2eac0b3d20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2e8806c8f0 0x7f2e8806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac160890 0x7f2eac160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 --2- 172.21.15.106:0/509776453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e900039b0 0x7f2eac15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.923+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 >> 172.21.15.106:0/509776453 conn(0x7f2eac0d0ae0 msgr2=0x7f2eac0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.924+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 shutdown_connections 2024-04-26T21:46:38.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:38.924+0000 7f2eabe00640 1 -- 172.21.15.106:0/509776453 wait complete. 2024-04-26T21:46:38.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:46:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.140+0000 7fd0dba00640 1 Processor -- start 2024-04-26T21:46:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.140+0000 7fd0dba00640 1 -- start start 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0dba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 0x7fd0d40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0dba00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d40af800 con 0x7fd0d415d100 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0dba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d40ae340 con 0x7fd0d40b5ca0 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0dba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d40afaa0 con 0x7fd0d406ce80 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0d9000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd0d415d100 0x7fd0d406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48514/0 (socket says 172.21.15.106:48514) 2024-04-26T21:46:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0d9000640 1 -- 172.21.15.106:0/200924791 learned_addr learned my addr 172.21.15.106:0/200924791 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:39.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329690668 0 0) 0x7fd0d40af800 con 0x7fd0d415d100 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0bc003520 con 0x7fd0d415d100 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.142+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 0x7fd0d40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1781313629 0 0) 0x7fd0bc003520 con 0x7fd0d415d100 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 msgr2=0x7fd0d40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 0x7fd0d40b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 >> v1:172.21.15.204:6789/0 conn(0x7fd0d40b5ca0 legacy=0x7fd0d40b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d40afd80 con 0x7fd0d415d100 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fd0c8001670 con 0x7fd0d415d100 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 >> v1:172.21.15.106:6789/0 conn(0x7fd0d415d100 legacy=0x7fd0d406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 0x7fd0bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0bc004040 0x7fd0bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc004d40 0x7fd0bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40afaa0 con 0x7fd0bc004040 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc00bbb0 con 0x7fd0bc004d40 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc00bdb0 con 0x7fd0d406ce80 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0bc004040 0x7fd0bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d3e00640 1 -- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 msgr2=0x7fd0bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.144+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0d406ce80 0x7fd0bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.144+0000 7fd0d3e00640 1 -- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc004d40 msgr2=0x7fd0bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.143+0000 7fd0d9a00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc004d40 0x7fd0bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.144+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc004d40 0x7fd0bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.144+0000 7fd0d3e00640 1 -- 172.21.15.106:0/200924791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc003520 con 0x7fd0bc004040 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.144+0000 7fd0d9a00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc004d40 0x7fd0bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0bc004040 0x7fd0bc004800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ba30 tx=0x7fd0bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773703019c726576 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0030070 con 0x7fd0bc004040 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c002ae10 con 0x7fd0bc004040 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0d3400640 1 -- 172.21.15.106:0/200924791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000b600 con 0x7fd0bc004040 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0dba00640 1 -- 172.21.15.106:0/200924791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0bc004040 msgr2=0x7fd0bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0dba00640 1 --2- 172.21.15.106:0/200924791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0bc004040 0x7fd0bc004800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ba30 tx=0x7fd0bc005830 comp rx=0 tx=0).stop 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0d9a00640 1 -- 172.21.15.106:0/200924791 reap_dead start 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0dba00640 1 -- 172.21.15.106:0/200924791 shutdown_connections 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.145+0000 7fd0dba00640 1 -- 172.21.15.106:0/200924791 >> 172.21.15.106:0/200924791 conn(0x7fd0d40b2d50 msgr2=0x7fd0d40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.146+0000 7fd0dba00640 1 -- 172.21.15.106:0/200924791 shutdown_connections 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.146+0000 7fd0dba00640 1 -- 172.21.15.106:0/200924791 wait complete. 2024-04-26T21:46:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.147+0000 7fd0dba00640 1 Processor -- start 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.147+0000 7fd0dba00640 1 -- start start 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 0x7fd0d4149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0d414aa80 0x7fd0d414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d409ee50 con 0x7fd0d415ed20 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b0340 con 0x7fd0d414aa80 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.148+0000 7fd0dba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d4160b70 con 0x7fd0bc004040 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57758/0 (socket says 172.21.15.106:57758) 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 -- 172.21.15.106:0/1422703845 learned_addr learned my addr 172.21.15.106:0/1422703845 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9000640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 0x7fd0d4149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d3e00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0d414aa80 0x7fd0d414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 -- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 msgr2=0x7fd0d4149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 0x7fd0d4149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 -- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0d414aa80 msgr2=0x7fd0d414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0d414aa80 0x7fd0d414a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 -- 172.21.15.106:0/1422703845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d415fd30 con 0x7fd0d415ed20 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9000640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 0x7fd0d4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d9a00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4073c70 tx=0x7fd0cc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.149+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc006b50 con 0x7fd0d415ed20 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.150+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0cc006cf0 con 0x7fd0d415ed20 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.150+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc004b60 con 0x7fd0d415ed20 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.150+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d4160010 con 0x7fd0d415ed20 2024-04-26T21:46:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.150+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d4148000 con 0x7fd0d415ed20 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.151+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4069650 con 0x7fd0d415ed20 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.152+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0cc019020 con 0x7fd0d415ed20 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.153+0000 7fd0d0c00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.153+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd0cc08b430 con 0x7fd0d415ed20 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.153+0000 7fd0d9000640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.154+0000 7fd0d9000640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 0x7fd0b406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8007a30 tx=0x7fd0c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.157+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0cc0596b0 con 0x7fd0d415ed20 2024-04-26T21:46:39.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.359+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd0d4073d00 con 0x7fd0d415ed20 2024-04-26T21:46:39.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.364+0000 7fd0d0c00640 1 -- 172.21.15.106:0/1422703845 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd0cc058fb0 con 0x7fd0d415ed20 2024-04-26T21:46:39.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.366+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 msgr2=0x7fd0b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.366+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 0x7fd0b406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8007a30 tx=0x7fd0c8009040 comp rx=0 tx=0).stop 2024-04-26T21:46:39.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 msgr2=0x7fd0d415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4073c70 tx=0x7fd0cc00dc00 comp rx=0 tx=0).stop 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 shutdown_connections 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d415ed20 0x7fd0d415f1a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0d414aa80 0x7fd0d414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 --2- 172.21.15.106:0/1422703845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0bc004040 0x7fd0d4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 >> 172.21.15.106:0/1422703845 conn(0x7fd0d40b2d50 msgr2=0x7fd0d40b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 shutdown_connections 2024-04-26T21:46:39.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.367+0000 7fd0dba00640 1 -- 172.21.15.106:0/1422703845 wait complete. 2024-04-26T21:46:39.386 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:46:39.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:46:39.487 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:46:39.606 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:46:39.606 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:39.606 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:39.664 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:39.665 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:39.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.917+0000 7f6772000640 1 Processor -- start 2024-04-26T21:46:39.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.917+0000 7f6772000640 1 -- start start 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6772000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6772000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f676c0d0300 0x7f676c0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6772000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c06de10 con 0x7f676c152780 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6772000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f676c06dbc0 con 0x7f676c0d0300 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6772000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f676c06e0f0 con 0x7f676c1523a0 2024-04-26T21:46:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6770c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.919+0000 7f6771600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f676c0d0300 0x7f676c0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.920+0000 7f6770c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c15c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:46140/0 (socket says 172.21.15.106:46140) 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.920+0000 7f6771600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f676c0d0300 0x7f676c0b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:52754/0 (socket says 172.21.15.106:52754) 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.920+0000 7f6770c00640 1 -- 172.21.15.106:0/422961963 learned_addr learned my addr 172.21.15.106:0/422961963 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.920+0000 7f6771600640 1 -- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 msgr2=0x7f676c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.920+0000 7f6771600640 1 --2- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.921+0000 7f6771600640 1 -- 172.21.15.106:0/422961963 >> v1:172.21.15.204:6789/0 conn(0x7f676c152780 legacy=0x7f676c0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.921+0000 7f6771600640 1 -- 172.21.15.106:0/422961963 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c06e3d0 con 0x7f676c0d0300 2024-04-26T21:46:39.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.922+0000 7f6771600640 1 --2- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c0d0300 0x7f676c0b4570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f676400bfa0 tx=0x7f676402c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232003eb8945178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f676b400640 1 -- 172.21.15.106:0/422961963 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764039070 con 0x7f676c0d0300 2024-04-26T21:46:39.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f676b400640 1 -- 172.21.15.106:0/422961963 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764009540 con 0x7f676c0d0300 2024-04-26T21:46:39.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f676b400640 1 -- 172.21.15.106:0/422961963 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764009910 con 0x7f676c0d0300 2024-04-26T21:46:39.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f6772000640 1 -- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c0d0300 msgr2=0x7f676c0b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f6772000640 1 --2- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c0d0300 0x7f676c0b4570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f676400bfa0 tx=0x7f676402c870 comp rx=0 tx=0).stop 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f6772000640 1 -- 172.21.15.106:0/422961963 shutdown_connections 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f6772000640 1 --2- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c0d0300 0x7f676c0b4570 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.928+0000 7f6772000640 1 --2- 172.21.15.106:0/422961963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.929+0000 7f6772000640 1 -- 172.21.15.106:0/422961963 >> 172.21.15.106:0/422961963 conn(0x7f676c06c740 msgr2=0x7f676c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.929+0000 7f6772000640 1 -- 172.21.15.106:0/422961963 shutdown_connections 2024-04-26T21:46:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.929+0000 7f6772000640 1 -- 172.21.15.106:0/422961963 wait complete. 2024-04-26T21:46:39.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.931+0000 7f6772000640 1 Processor -- start 2024-04-26T21:46:39.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.931+0000 7f6772000640 1 -- start start 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c1e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 0x7f676c1e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0b4ab0 con 0x7f676c0d0300 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0d0730 con 0x7f676c152780 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6772000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f676c0b4c30 con 0x7f676c1523a0 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6770c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6770c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57762/0 (socket says 172.21.15.106:57762) 2024-04-26T21:46:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6770c00640 1 -- 172.21.15.106:0/2184420759 learned_addr learned my addr 172.21.15.106:0/2184420759 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f6771600640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 0x7f676c1e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.932+0000 7f676be00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c1e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 -- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 msgr2=0x7f676c1e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c1e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 -- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 msgr2=0x7f676c1e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 0x7f676c1e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 -- 172.21.15.106:0/2184420759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0ced30 con 0x7f676c0d0300 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6771600640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 0x7f676c1e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:39.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f676be00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c1e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6770c00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f676000f980 tx=0x7f676000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760019070 con 0x7f676c0d0300 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676000dc50 con 0x7f676c0d0300 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c0cf010 con 0x7f676c0d0300 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.933+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676000b040 con 0x7f676c0d0300 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.934+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1e4930 con 0x7f676c0d0300 2024-04-26T21:46:39.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.935+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c0693c0 con 0x7f676c0d0300 2024-04-26T21:46:39.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.936+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f676001c070 con 0x7f676c0d0300 2024-04-26T21:46:39.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.937+0000 7f6768c00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 0x7f674c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.937+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f676008b9c0 con 0x7f676c0d0300 2024-04-26T21:46:39.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.937+0000 7f676be00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 0x7f674c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.938+0000 7f676be00640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 0x7f674c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f676c1f84e0 tx=0x7f6758009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:39.942+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f676008de20 con 0x7f676c0d0300 2024-04-26T21:46:40.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.141+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f676c0721c0 con 0x7f676c0d0300 2024-04-26T21:46:40.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.143+0000 7f6768c00640 1 -- 172.21.15.106:0/2184420759 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f6760059d70 con 0x7f676c0d0300 2024-04-26T21:46:40.143 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:40.144 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:40.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 msgr2=0x7f674c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 0x7f674c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f676c1f84e0 tx=0x7f6758009040 comp rx=0 tx=0).stop 2024-04-26T21:46:40.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 msgr2=0x7f676c1f4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f676000f980 tx=0x7f676000fe40 comp rx=0 tx=0).stop 2024-04-26T21:46:40.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 shutdown_connections 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c152780 0x7f676c1e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f674c06c920 0x7f674c06ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f676c1523a0 0x7f676c1e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 --2- 172.21.15.106:0/2184420759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f676c0d0300 0x7f676c1f4990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 >> 172.21.15.106:0/2184420759 conn(0x7f676c06c740 msgr2=0x7f676c1513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 shutdown_connections 2024-04-26T21:46:40.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.146+0000 7f6772000640 1 -- 172.21.15.106:0/2184420759 wait complete. 2024-04-26T21:46:40.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:40.167 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.366+0000 7f37c5c00640 1 Processor -- start 2024-04-26T21:46:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.366+0000 7f37c5c00640 1 -- start start 2024-04-26T21:46:40.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37c00b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37c5c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c006eea0 con 0x7f37c006c770 2024-04-26T21:46:40.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37c5c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c006d080 con 0x7f37c0156650 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37c5c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37c006f140 con 0x7f37c006cbf0 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37b7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f37c006c770 0x7f37c00b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48528/0 (socket says 172.21.15.106:48528) 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37b7e00640 1 -- 172.21.15.106:0/2348515805 learned_addr learned my addr 172.21.15.106:0/2348515805 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37bf400640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37c00b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613926453 0 0) 0x7f37c006eea0 con 0x7f37c006c770 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.368+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a4003520 con 0x7f37c006c770 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2095208190 0 0) 0x7f37a4003520 con 0x7f37c006c770 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 msgr2=0x7f37c00b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37c00b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 >> v1:172.21.15.204:6789/0 conn(0x7f37c0156650 legacy=0x7f37c015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c006f420 con 0x7f37c006c770 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f37a8001670 con 0x7f37c006c770 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 >> v1:172.21.15.106:6789/0 conn(0x7f37c006c770 legacy=0x7f37c00b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37a40039e0 0x7f37a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a4004800 0x7f37a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37a4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37c006f140 con 0x7f37a4004800 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400bb50 con 0x7f37a40039e0 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37a400bd50 con 0x7f37c006cbf0 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.369+0000 7f37b7e00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a4004800 0x7f37a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37bea00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37a40039e0 0x7f37a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37bf400640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37a4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37b7e00640 1 -- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 msgr2=0x7f37a4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37b7e00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c006cbf0 0x7f37a4005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37b7e00640 1 -- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37a40039e0 msgr2=0x7f37a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37b7e00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37a40039e0 0x7f37a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37b7e00640 1 -- 172.21.15.106:0/2348515805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a4003520 con 0x7f37a4004800 2024-04-26T21:46:40.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.370+0000 7f37bea00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37a40039e0 0x7f37a4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.371+0000 7f37b7e00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a4004800 0x7f37a4004c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c190 tx=0x7f37a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ba982f59d157ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.371+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8007c60 con 0x7f37a4004800 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.371+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a8005ad0 con 0x7f37a4004800 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.371+0000 7f37be000640 1 -- 172.21.15.106:0/2348515805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8007070 con 0x7f37a4004800 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 -- 172.21.15.106:0/2348515805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a4004800 msgr2=0x7f37a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 --2- 172.21.15.106:0/2348515805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a4004800 0x7f37a4004c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c190 tx=0x7f37a4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37bf400640 1 -- 172.21.15.106:0/2348515805 reap_dead start 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 -- 172.21.15.106:0/2348515805 shutdown_connections 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 -- 172.21.15.106:0/2348515805 >> 172.21.15.106:0/2348515805 conn(0x7f37c00b2d50 msgr2=0x7f37c00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 -- 172.21.15.106:0/2348515805 shutdown_connections 2024-04-26T21:46:40.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.372+0000 7f37c5c00640 1 -- 172.21.15.106:0/2348515805 wait complete. 2024-04-26T21:46:40.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.374+0000 7f37c5c00640 1 Processor -- start 2024-04-26T21:46:40.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.374+0000 7f37c5c00640 1 -- start start 2024-04-26T21:46:40.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37a4004800 0x7f37c01f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 0x7f37c00cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37c006fcc0 con 0x7f37c0142ef0 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37c006f9e0 con 0x7f37c0072a60 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37c5c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37c015c680 con 0x7f37a4004800 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57782/0 (socket says 172.21.15.106:57782) 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 -- 172.21.15.106:0/12760740 learned_addr learned my addr 172.21.15.106:0/12760740 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37bf400640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 0x7f37c00cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 -- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37a4004800 msgr2=0x7f37c01f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37bea00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37a4004800 0x7f37c01f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37a4004800 0x7f37c01f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 -- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 msgr2=0x7f37c00cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 0x7f37c00cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37b7e00640 1 -- 172.21.15.106:0/12760740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c00cd720 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.375+0000 7f37bf400640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 0x7f37c00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37b7e00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c280 tx=0x7f37a8001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8037040 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a8041070 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80015b0 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c00cda00 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.376+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01f92e0 con 0x7f37c0142ef0 2024-04-26T21:46:40.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.377+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0069650 con 0x7f37c0142ef0 2024-04-26T21:46:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.378+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37a8007070 con 0x7f37c0142ef0 2024-04-26T21:46:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.379+0000 7f37b6a00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 0x7f379c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.379+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f37a80abab0 con 0x7f37c0142ef0 2024-04-26T21:46:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.379+0000 7f37bea00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 0x7f379c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.380+0000 7f37bea00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 0x7f379c06ed80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007fd0 tx=0x7f37b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.384+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37a8079dc0 con 0x7f37c0142ef0 2024-04-26T21:46:40.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.582+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37c0072e40 con 0x7f37c0142ef0 2024-04-26T21:46:40.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.583+0000 7f37b6a00640 1 -- 172.21.15.106:0/12760740 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f37a80796c0 con 0x7f37c0142ef0 2024-04-26T21:46:40.583 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 msgr2=0x7f379c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 0x7f379c06ed80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007fd0 tx=0x7f37b0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 msgr2=0x7f37c0072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c280 tx=0x7f37a8001970 comp rx=0 tx=0).stop 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 shutdown_connections 2024-04-26T21:46:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0072a60 0x7f37c00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c0142ef0 0x7f37c0072520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379c06c8c0 0x7f379c06ed80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 --2- 172.21.15.106:0/12760740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37a4004800 0x7f37c01f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 >> 172.21.15.106:0/12760740 conn(0x7f37c00b2d50 msgr2=0x7f37c015edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 shutdown_connections 2024-04-26T21:46:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.586+0000 7f37c5c00640 1 -- 172.21.15.106:0/12760740 wait complete. 2024-04-26T21:46:40.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:40.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.809+0000 7f4696800640 1 Processor -- start 2024-04-26T21:46:40.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.809+0000 7f4696800640 1 -- start start 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4696800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46900721b0 0x7f4690163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4696800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469006e080 con 0x7f46901640f0 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4696800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469006e360 con 0x7f469014cef0 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4696800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f469006e640 con 0x7f46900721b0 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4695e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f46901640f0 0x7f46900b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48544/0 (socket says 172.21.15.106:48544) 2024-04-26T21:46:40.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.811+0000 7f4695e00640 1 -- 172.21.15.106:0/3676853178 learned_addr learned my addr 172.21.15.106:0/3676853178 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:40.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f4694a00640 1 --2- 172.21.15.106:0/3676853178 >> v2:172.21.15.204:3300/0 conn(0x7f46900721b0 0x7f4690163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139875461 0 0) 0x7f469006e080 con 0x7f46901640f0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4678003520 con 0x7f46901640f0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 381529283 0 0) 0x7f4678003520 con 0x7f46901640f0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 >> v2:172.21.15.204:3300/0 conn(0x7f46900721b0 msgr2=0x7f4690163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 --2- 172.21.15.106:0/3676853178 >> v2:172.21.15.204:3300/0 conn(0x7f46900721b0 0x7f4690163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 >> v1:172.21.15.204:6789/0 conn(0x7f469014cef0 legacy=0x7f46900730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469014d2d0 con 0x7f46901640f0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f46880019f0 con 0x7f46901640f0 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 >> v1:172.21.15.106:6789/0 conn(0x7f46901640f0 legacy=0x7f46900b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4678003a20 0x7f4678003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f469006e640 con 0x7f4678004800 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f467800c0f0 con 0x7f46780051c0 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.812+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f467800c2f0 con 0x7f4678003a20 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4695400640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4678003a20 0x7f4678003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4695e00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 -- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4678003a20 msgr2=0x7f4678003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4678003a20 0x7f4678003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 -- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46780051c0 msgr2=0x7f467800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4694a00640 1 -- 172.21.15.106:0/3676853178 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4678003520 con 0x7f4678004800 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4695e00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.813+0000 7f4695e00640 1 -- 172.21.15.106:0/3676853178 reap_dead start 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.814+0000 7f4694a00640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f4678004c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f467c009ac0 tx=0x7f4678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f0cded1afe9659 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.814+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c036070 con 0x7f4678004800 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.814+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c031400 con 0x7f4678004800 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.814+0000 7f468fe00640 1 -- 172.21.15.106:0/3676853178 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c009690 con 0x7f4678004800 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 -- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 msgr2=0x7f4678004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f4678004c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f467c009ac0 tx=0x7f4678005830 comp rx=0 tx=0).stop 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 -- 172.21.15.106:0/3676853178 shutdown_connections 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 --2- 172.21.15.106:0/3676853178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 -- 172.21.15.106:0/3676853178 >> 172.21.15.106:0/3676853178 conn(0x7f46900b2900 msgr2=0x7f46900b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 -- 172.21.15.106:0/3676853178 shutdown_connections 2024-04-26T21:46:40.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.815+0000 7f4696800640 1 -- 172.21.15.106:0/3676853178 wait complete. 2024-04-26T21:46:40.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.817+0000 7f4696800640 1 Processor -- start 2024-04-26T21:46:40.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.817+0000 7f4696800640 1 -- start start 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4690163210 0x7f4690145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 0x7f46900c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4690163f40 con 0x7f4678004800 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4690163dc0 con 0x7f4690147440 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4696800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46900730a0 con 0x7f4690163210 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4695400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4695400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57806/0 (socket says 172.21.15.106:57806) 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.818+0000 7f4695400640 1 -- 172.21.15.106:0/2441338229 learned_addr learned my addr 172.21.15.106:0/2441338229 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4694a00640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4690163210 0x7f4690145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 -- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4690163210 msgr2=0x7f4690145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4690163210 0x7f4690145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695e00640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 0x7f46900c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 -- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 msgr2=0x7f46900c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 0x7f46900c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 -- 172.21.15.106:0/2441338229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46901463a0 con 0x7f4678004800 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695e00640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 0x7f46900c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4695400640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f468400e440 tx=0x7f468400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684019070 con 0x7f4678004800 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46840052e0 con 0x7f4678004800 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684003810 con 0x7f4678004800 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.819+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4690146680 con 0x7f4678004800 2024-04-26T21:46:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.820+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4690146c40 con 0x7f4678004800 2024-04-26T21:46:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.821+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46900693c0 con 0x7f4678004800 2024-04-26T21:46:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.822+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f468401c070 con 0x7f4678004800 2024-04-26T21:46:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.823+0000 7f468d600640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 0x7f467006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.823+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f468408b060 con 0x7f4678004800 2024-04-26T21:46:40.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.823+0000 7f4694a00640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 0x7f467006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.824+0000 7f4694a00640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 0x7f467006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f467c033000 tx=0x7f467c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:40.827+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4684058dc0 con 0x7f4678004800 2024-04-26T21:46:41.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.026+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4690165a40 con 0x7f4678004800 2024-04-26T21:46:41.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.027+0000 7f468d600640 1 -- 172.21.15.106:0/2441338229 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f468405d0c0 con 0x7f4678004800 2024-04-26T21:46:41.027 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.029+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 msgr2=0x7f467006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.029+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 0x7f467006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f467c033000 tx=0x7f467c008030 comp rx=0 tx=0).stop 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.029+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 msgr2=0x7f469014a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.029+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f468400e440 tx=0x7f468400e900 comp rx=0 tx=0).stop 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 shutdown_connections 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4690147440 0x7f46900c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f467006c920 0x7f467006ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4690163210 0x7f4690145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 --2- 172.21.15.106:0/2441338229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4678004800 0x7f469014a0f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 >> 172.21.15.106:0/2441338229 conn(0x7f46900b2900 msgr2=0x7f46900d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 shutdown_connections 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.030+0000 7f4696800640 1 -- 172.21.15.106:0/2441338229 wait complete. 2024-04-26T21:46:41.048 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:41.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.246+0000 7f2503800640 1 Processor -- start 2024-04-26T21:46:41.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.247+0000 7f2503800640 1 -- start start 2024-04-26T21:46:41.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.248+0000 7f2503800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24fc147f20 0x7f24fc0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.248+0000 7f2503800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24fc0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.248+0000 7f2503800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24fc0ade20 con 0x7f24fc073da0 2024-04-26T21:46:41.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.248+0000 7f2503800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f24fc0ae100 con 0x7f24fc147f20 2024-04-26T21:46:41.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.248+0000 7f2503800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc0ae3e0 con 0x7f24fc06ce80 2024-04-26T21:46:41.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f24f3e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f24fc073da0 0x7f24fc06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48548/0 (socket says 172.21.15.106:48548) 2024-04-26T21:46:41.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f24f3e00640 1 -- 172.21.15.106:0/3532690476 learned_addr learned my addr 172.21.15.106:0/3532690476 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f2501800640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24fc0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f2500e00640 1 --2- 172.21.15.106:0/3532690476 >> v2:172.21.15.204:3300/0 conn(0x7f24fc147f20 0x7f24fc0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777555085 0 0) 0x7f24fc0ade20 con 0x7f24fc073da0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e0003520 con 0x7f24fc073da0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.249+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4058177210 0 0) 0x7f24e0003520 con 0x7f24fc073da0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 msgr2=0x7f24fc0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24fc0aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 >> v2:172.21.15.204:3300/0 conn(0x7f24fc147f20 msgr2=0x7f24fc0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 --2- 172.21.15.106:0/3532690476 >> v2:172.21.15.204:3300/0 conn(0x7f24fc147f20 0x7f24fc0b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc0ae6c0 con 0x7f24fc073da0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f24e4001670 con 0x7f24fc073da0 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 >> v1:172.21.15.106:6789/0 conn(0x7f24fc073da0 legacy=0x7f24fc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e00039b0 0x7f24e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24e0004800 0x7f24e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc0ae100 con 0x7f24e0004800 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc0ae3e0 con 0x7f24e00039b0 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24e000baa0 con 0x7f24fc06ce80 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24f3e00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24e0004800 0x7f24e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f2500e00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e00039b0 0x7f24e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f2501800640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24f3e00640 1 -- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 msgr2=0x7f24e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.250+0000 7f24f3e00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.251+0000 7f24f3e00640 1 -- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e00039b0 msgr2=0x7f24e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.251+0000 7f24f3e00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e00039b0 0x7f24e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.251+0000 7f24f3e00640 1 -- 172.21.15.106:0/3532690476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e0003520 con 0x7f24e0004800 2024-04-26T21:46:41.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.251+0000 7f2501800640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24fc06ce80 0x7f24e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:41.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f24f3e00640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24e0004800 0x7f24e0004c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c210 tx=0x7f24e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fbc72153b1a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4007d30 con 0x7f24e0004800 2024-04-26T21:46:41.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e40058d0 con 0x7f24e0004800 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f24fbe00640 1 -- 172.21.15.106:0/3532690476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4005ca0 con 0x7f24e0004800 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f2503800640 1 -- 172.21.15.106:0/3532690476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24e0004800 msgr2=0x7f24e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f2503800640 1 --2- 172.21.15.106:0/3532690476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24e0004800 0x7f24e0004c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c210 tx=0x7f24e0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.252+0000 7f2501800640 1 -- 172.21.15.106:0/3532690476 reap_dead start 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.253+0000 7f2503800640 1 -- 172.21.15.106:0/3532690476 shutdown_connections 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.253+0000 7f2503800640 1 -- 172.21.15.106:0/3532690476 >> 172.21.15.106:0/3532690476 conn(0x7f24fc0b2d50 msgr2=0x7f24fc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.253+0000 7f2503800640 1 -- 172.21.15.106:0/3532690476 shutdown_connections 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.253+0000 7f2503800640 1 -- 172.21.15.106:0/3532690476 wait complete. 2024-04-26T21:46:41.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.254+0000 7f2503800640 1 Processor -- start 2024-04-26T21:46:41.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.255+0000 7f2503800640 1 -- start start 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.255+0000 7f2503800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24e0004800 0x7f24fc15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2503800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2503800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24fc15fd20 0x7f24fc0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2503800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc06e220 con 0x7f24fc141880 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2503800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc074230 con 0x7f24fc15fd20 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2503800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc0aeec0 con 0x7f24e0004800 2024-04-26T21:46:41.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57828/0 (socket says 172.21.15.106:57828) 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 -- 172.21.15.106:0/2670653926 learned_addr learned my addr 172.21.15.106:0/2670653926 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2500e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24e0004800 0x7f24fc15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f2501800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24fc15fd20 0x7f24fc0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 -- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24e0004800 msgr2=0x7f24fc15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24e0004800 0x7f24fc15ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 -- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24fc15fd20 msgr2=0x7f24fc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24fc15fd20 0x7f24fc0cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.256+0000 7f24f3e00640 1 -- 172.21.15.106:0/2670653926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc15ef40 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f24f3e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c330 tx=0x7f24fc0adff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4043070 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e4007070 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4038040 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc0cebd0 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.257+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc0cf190 con 0x7f24fc141880 2024-04-26T21:46:41.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.258+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc069650 con 0x7f24fc141880 2024-04-26T21:46:41.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.259+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e4046070 con 0x7f24fc141880 2024-04-26T21:46:41.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.260+0000 7f24f9600640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 0x7f24d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.260+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f24e40ab7c0 con 0x7f24fc141880 2024-04-26T21:46:41.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.260+0000 7f2500e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 0x7f24d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.261+0000 7f2500e00640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 0x7f24d806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006fd0 tx=0x7f24ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.265+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24fc15ef40 con 0x7f24fc141880 2024-04-26T21:46:41.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.461+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24fc0af970 con 0x7f24fc141880 2024-04-26T21:46:41.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.462+0000 7f24f9600640 1 -- 172.21.15.106:0/2670653926 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f24e4079a40 con 0x7f24fc141880 2024-04-26T21:46:41.463 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 msgr2=0x7f24d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 0x7f24d806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006fd0 tx=0x7f24ec008040 comp rx=0 tx=0).stop 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 msgr2=0x7f24fc15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c330 tx=0x7f24fc0adff0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 shutdown_connections 2024-04-26T21:46:41.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24fc15fd20 0x7f24fc0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24fc141880 0x7f24fc15f7e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f24d806c920 0x7f24d806ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 --2- 172.21.15.106:0/2670653926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24e0004800 0x7f24fc15ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 >> 172.21.15.106:0/2670653926 conn(0x7f24fc0b2d50 msgr2=0x7f24fc0b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 shutdown_connections 2024-04-26T21:46:41.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.465+0000 7f2503800640 1 -- 172.21.15.106:0/2670653926 wait complete. 2024-04-26T21:46:41.484 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:41.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.682+0000 7f3126c00640 1 Processor -- start 2024-04-26T21:46:41.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.683+0000 7f3126c00640 1 -- start start 2024-04-26T21:46:41.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.684+0000 7f3126c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f312006fb50 0x7f31200730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.684+0000 7f3126c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f312006ff70 0x7f31200735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.684+0000 7f3126c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31200ae5f0 con 0x7f312006e130 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.684+0000 7f3126c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3120072010 con 0x7f312006fb50 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.684+0000 7f3126c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f31200722f0 con 0x7f312006ff70 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f312006fb50 0x7f31200730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f312006fb50 0x7f31200730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57832/0 (socket says 172.21.15.106:57832) 2024-04-26T21:46:41.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 -- 172.21.15.106:0/1852417092 learned_addr learned my addr 172.21.15.106:0/1852417092 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:41.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3124c00640 1 --2- 172.21.15.106:0/1852417092 >> v2:172.21.15.204:3300/0 conn(0x7f312006ff70 0x7f31200735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 -- 172.21.15.106:0/1852417092 >> v2:172.21.15.204:3300/0 conn(0x7f312006ff70 msgr2=0x7f31200735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 --2- 172.21.15.106:0/1852417092 >> v2:172.21.15.204:3300/0 conn(0x7f312006ff70 0x7f31200735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 -- 172.21.15.106:0/1852417092 >> v1:172.21.15.204:6789/0 conn(0x7f312006e130 legacy=0x7f31200ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.685+0000 7f3117400640 1 -- 172.21.15.106:0/1852417092 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200725d0 con 0x7f312006fb50 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.686+0000 7f3117400640 1 --2- 172.21.15.106:0/1852417092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006fb50 0x7f31200730b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f31200b4350 tx=0x7f310802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d4f6ccd7dda4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.686+0000 7f311f400640 1 -- 172.21.15.106:0/1852417092 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108037070 con 0x7f312006fb50 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.686+0000 7f311f400640 1 -- 172.21.15.106:0/1852417092 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3108032440 con 0x7f312006fb50 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.686+0000 7f311f400640 1 -- 172.21.15.106:0/1852417092 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108032810 con 0x7f312006fb50 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 -- 172.21.15.106:0/1852417092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006fb50 msgr2=0x7f31200730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 --2- 172.21.15.106:0/1852417092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006fb50 0x7f31200730b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f31200b4350 tx=0x7f310802a230 comp rx=0 tx=0).stop 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 -- 172.21.15.106:0/1852417092 shutdown_connections 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 --2- 172.21.15.106:0/1852417092 >> v2:172.21.15.204:3300/0 conn(0x7f312006ff70 0x7f31200735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 --2- 172.21.15.106:0/1852417092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006fb50 0x7f31200730b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 -- 172.21.15.106:0/1852417092 >> 172.21.15.106:0/1852417092 conn(0x7f31200b2d50 msgr2=0x7f31200b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 -- 172.21.15.106:0/1852417092 shutdown_connections 2024-04-26T21:46:41.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.687+0000 7f3126c00640 1 -- 172.21.15.106:0/1852417092 wait complete. 2024-04-26T21:46:41.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.689+0000 7f3126c00640 1 Processor -- start 2024-04-26T21:46:41.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.689+0000 7f3126c00640 1 -- start start 2024-04-26T21:46:41.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f312006e130 0x7f3120147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312006fb50 0x7f312015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31200b4810 con 0x7f312006ff70 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3120072b90 con 0x7f312006fb50 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3126c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31200b4990 con 0x7f312006e130 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3124c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3124c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57846/0 (socket says 172.21.15.106:57846) 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.690+0000 7f3124c00640 1 -- 172.21.15.106:0/2424586064 learned_addr learned my addr 172.21.15.106:0/2424586064 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f311fe00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f312006e130 0x7f3120147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3117400640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312006fb50 0x7f312015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 -- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f312006e130 msgr2=0x7f3120147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f312006e130 0x7f3120147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 -- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312006fb50 msgr2=0x7f312015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312006fb50 0x7f312015a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 -- 172.21.15.106:0/2424586064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312015afe0 con 0x7f312006ff70 2024-04-26T21:46:41.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3124c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f311800cd70 tx=0x7f311800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118003b90 con 0x7f312006ff70 2024-04-26T21:46:41.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3118003d30 con 0x7f312006ff70 2024-04-26T21:46:41.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118004240 con 0x7f312006ff70 2024-04-26T21:46:41.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201593e0 con 0x7f312006ff70 2024-04-26T21:46:41.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.691+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201599a0 con 0x7f312006ff70 2024-04-26T21:46:41.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.693+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3120069540 con 0x7f312006ff70 2024-04-26T21:46:41.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.693+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31180043e0 con 0x7f312006ff70 2024-04-26T21:46:41.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.694+0000 7f311cc00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.694+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f311808c710 con 0x7f312006ff70 2024-04-26T21:46:41.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.695+0000 7f311fe00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.695+0000 7f311fe00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 0x7f30fc06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f311000a730 tx=0x7f3110008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.699+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f311801d340 con 0x7f312006ff70 2024-04-26T21:46:41.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.913+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3120154090 con 0x7f312006ff70 2024-04-26T21:46:41.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.914+0000 7f311cc00640 1 -- 172.21.15.106:0/2424586064 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f311805aab0 con 0x7f312006ff70 2024-04-26T21:46:41.914 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:41.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 msgr2=0x7f30fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 0x7f30fc06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f311000a730 tx=0x7f3110008040 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 msgr2=0x7f3120153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f311800cd70 tx=0x7f311800a660 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 shutdown_connections 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f312006ff70 0x7f3120153130 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312006fb50 0x7f312015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30fc06c8f0 0x7f30fc06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 --2- 172.21.15.106:0/2424586064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f312006e130 0x7f3120147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 >> 172.21.15.106:0/2424586064 conn(0x7f31200b2d50 msgr2=0x7f31200b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 shutdown_connections 2024-04-26T21:46:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:41.917+0000 7f3126c00640 1 -- 172.21.15.106:0/2424586064 wait complete. 2024-04-26T21:46:41.939 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:41.939 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:41.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:42.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.225+0000 7f4d8ec00640 1 Processor -- start 2024-04-26T21:46:42.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.225+0000 7f4d8ec00640 1 -- start start 2024-04-26T21:46:42.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ec00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d880d0e70 0x7f4d880d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 0x7f4d880b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d88146a60 con 0x7f4d88146680 2024-04-26T21:46:42.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ec00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8806e970 con 0x7f4d880d0e70 2024-04-26T21:46:42.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880b60a0 con 0x7f4d880b3b20 2024-04-26T21:46:42.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ce00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d880d0e70 0x7f4d880d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ce00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d880d0e70 0x7f4d880d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57854/0 (socket says 172.21.15.106:57854) 2024-04-26T21:46:42.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8ce00640 1 -- 172.21.15.106:0/1340472571 learned_addr learned my addr 172.21.15.106:0/1340472571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.227+0000 7f4d8e200640 1 --2- 172.21.15.106:0/1340472571 >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 0x7f4d880b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.228+0000 7f4d8ce00640 1 -- 172.21.15.106:0/1340472571 >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 msgr2=0x7f4d880b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.228+0000 7f4d8ce00640 1 --2- 172.21.15.106:0/1340472571 >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 0x7f4d880b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.228+0000 7f4d8ce00640 1 -- 172.21.15.106:0/1340472571 >> v1:172.21.15.204:6789/0 conn(0x7f4d88146680 legacy=0x7f4d880b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.228+0000 7f4d8ce00640 1 -- 172.21.15.106:0/1340472571 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8806dd60 con 0x7f4d880d0e70 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.228+0000 7f4d8e200640 1 --2- 172.21.15.106:0/1340472571 >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 0x7f4d880b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.229+0000 7f4d8ce00640 1 --2- 172.21.15.106:0/1340472571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d880d12b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4d8806f370 tx=0x7f4d7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7523e5573ae7e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.229+0000 7f4d87e00640 1 -- 172.21.15.106:0/1340472571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74037070 con 0x7f4d880d0e70 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.229+0000 7f4d87e00640 1 -- 172.21.15.106:0/1340472571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d74032440 con 0x7f4d880d0e70 2024-04-26T21:46:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.229+0000 7f4d87e00640 1 -- 172.21.15.106:0/1340472571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74032810 con 0x7f4d880d0e70 2024-04-26T21:46:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 -- 172.21.15.106:0/1340472571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 msgr2=0x7f4d880d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/1340472571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d880d12b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4d8806f370 tx=0x7f4d7402a230 comp rx=0 tx=0).stop 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 -- 172.21.15.106:0/1340472571 shutdown_connections 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/1340472571 >> v2:172.21.15.204:3300/0 conn(0x7f4d880b3b20 0x7f4d880b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/1340472571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d880d12b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 -- 172.21.15.106:0/1340472571 >> 172.21.15.106:0/1340472571 conn(0x7f4d8806c740 msgr2=0x7f4d8806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 -- 172.21.15.106:0/1340472571 shutdown_connections 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.230+0000 7f4d8ec00640 1 -- 172.21.15.106:0/1340472571 wait complete. 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.231+0000 7f4d8ec00640 1 Processor -- start 2024-04-26T21:46:42.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.232+0000 7f4d8ec00640 1 -- start start 2024-04-26T21:46:42.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.232+0000 7f4d8ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d880b3b20 0x7f4d881f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.232+0000 7f4d8ec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 0x7f4d88137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8806e320 con 0x7f4d880d0e70 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880d17f0 con 0x7f4d88146680 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8806e600 con 0x7f4d880b3b20 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57860/0 (socket says 172.21.15.106:57860) 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 -- 172.21.15.106:0/2030855165 learned_addr learned my addr 172.21.15.106:0/2030855165 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 -- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d880b3b20 msgr2=0x7f4d881f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d880b3b20 0x7f4d881f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8e200640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 0x7f4d88137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 -- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 msgr2=0x7f4d88137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 0x7f4d88137f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8ce00640 1 -- 172.21.15.106:0/2030855165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88154cb0 con 0x7f4d880d0e70 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.233+0000 7f4d8e200640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 0x7f4d88137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:42.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d8ce00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d7402a270 tx=0x7f4d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74037040 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d74042070 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74030bb0 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8813ac00 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.234+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881f8690 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.235+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880693c0 con 0x7f4d880d0e70 2024-04-26T21:46:42.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.236+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d74047460 con 0x7f4d880d0e70 2024-04-26T21:46:42.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.237+0000 7f4d85600640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.237+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4d740ac880 con 0x7f4d880d0e70 2024-04-26T21:46:42.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.237+0000 7f4d8d800640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.238+0000 7f4d8d800640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009650 tx=0x7f4d7c006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.242+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d74045360 con 0x7f4d880d0e70 2024-04-26T21:46:42.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.446+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d880b5e60 con 0x7f4d880d0e70 2024-04-26T21:46:42.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.450+0000 7f4d85600640 1 -- 172.21.15.106:0/2030855165 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f4d7407ac20 con 0x7f4d880d0e70 2024-04-26T21:46:42.450 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:42.450 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.452+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 msgr2=0x7f4d6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.452+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009650 tx=0x7f4d7c006ba0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.452+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 msgr2=0x7f4d881f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.452+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d7402a270 tx=0x7f4d740046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 shutdown_connections 2024-04-26T21:46:42.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88146680 0x7f4d88137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d880d0e70 0x7f4d881f4830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 --2- 172.21.15.106:0/2030855165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d880b3b20 0x7f4d881f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 >> 172.21.15.106:0/2030855165 conn(0x7f4d8806c740 msgr2=0x7f4d8806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 shutdown_connections 2024-04-26T21:46:42.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.453+0000 7f4d8ec00640 1 -- 172.21.15.106:0/2030855165 wait complete. 2024-04-26T21:46:42.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:42.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:42.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.679+0000 7f9ed5a00640 1 Processor -- start 2024-04-26T21:46:42.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.679+0000 7f9ed5a00640 1 -- start start 2024-04-26T21:46:42.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.681+0000 7f9ed5a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ed0072970 0x7f9ed00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.681+0000 7f9ed5a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed006e320 con 0x7f9ed00b4ed0 2024-04-26T21:46:42.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.681+0000 7f9ed5a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed006e600 con 0x7f9ed0071e80 2024-04-26T21:46:42.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.681+0000 7f9ed5a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed006e8e0 con 0x7f9ed0072970 2024-04-26T21:46:42.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.682+0000 7f9ed5000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ed0072970 0x7f9ed00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.682+0000 7f9ecf400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9ed00b4ed0 0x7f9ed006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48562/0 (socket says 172.21.15.106:48562) 2024-04-26T21:46:42.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.682+0000 7f9ecf400640 1 -- 172.21.15.106:0/2300284054 learned_addr learned my addr 172.21.15.106:0/2300284054 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:42.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.682+0000 7f9ed5000640 1 -- 172.21.15.106:0/2300284054 >> v1:172.21.15.204:6789/0 conn(0x7f9ed0071e80 legacy=0x7f9ed0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.682+0000 7f9ed5000640 1 -- 172.21.15.106:0/2300284054 >> v1:172.21.15.106:6789/0 conn(0x7f9ed00b4ed0 legacy=0x7f9ed006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.683+0000 7f9ecea00640 1 -- 172.21.15.106:0/2300284054 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715241463 0 0) 0x7f9ed006e320 con 0x7f9ed00b4ed0 2024-04-26T21:46:42.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.683+0000 7f9ed5000640 1 -- 172.21.15.106:0/2300284054 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed006ebc0 con 0x7f9ed0072970 2024-04-26T21:46:42.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ed5000640 1 --2- 172.21.15.106:0/2300284054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0072970 0x7f9ed00b3ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2581a2542dfeff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ecea00640 1 -- 172.21.15.106:0/2300284054 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007740 con 0x7f9ed0072970 2024-04-26T21:46:42.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ecea00640 1 -- 172.21.15.106:0/2300284054 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec80078e0 con 0x7f9ed0072970 2024-04-26T21:46:42.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ecea00640 1 -- 172.21.15.106:0/2300284054 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007cb0 con 0x7f9ed0072970 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ed5a00640 1 -- 172.21.15.106:0/2300284054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0072970 msgr2=0x7f9ed00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.684+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/2300284054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0072970 0x7f9ed00b3ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.685+0000 7f9ed5a00640 1 -- 172.21.15.106:0/2300284054 shutdown_connections 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.685+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/2300284054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0072970 0x7f9ed00b3ee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.685+0000 7f9ed5a00640 1 -- 172.21.15.106:0/2300284054 >> 172.21.15.106:0/2300284054 conn(0x7f9ed00b2900 msgr2=0x7f9ed00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.685+0000 7f9ed5a00640 1 -- 172.21.15.106:0/2300284054 shutdown_connections 2024-04-26T21:46:42.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.685+0000 7f9ed5a00640 1 -- 172.21.15.106:0/2300284054 wait complete. 2024-04-26T21:46:42.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.686+0000 7f9ed5a00640 1 Processor -- start 2024-04-26T21:46:42.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 -- start start 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed0072970 0x7f9ed00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ed00b4ed0 0x7f9ed0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00ae8c0 con 0x7f9ed0071e80 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00b5ff0 con 0x7f9ed0072970 2024-04-26T21:46:42.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.687+0000 7f9ed5a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00aea40 con 0x7f9ed00b4ed0 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57880/0 (socket says 172.21.15.106:57880) 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 -- 172.21.15.106:0/1390088953 learned_addr learned my addr 172.21.15.106:0/1390088953 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 -- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ed00b4ed0 msgr2=0x7f9ed0157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ed00b4ed0 0x7f9ed0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 -- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed0072970 msgr2=0x7f9ed00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed0072970 0x7f9ed00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 -- 172.21.15.106:0/1390088953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed01619c0 con 0x7f9ed0071e80 2024-04-26T21:46:42.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.688+0000 7f9ecfe00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c480 tx=0x7f9ec000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.689+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00052d0 con 0x7f9ed0071e80 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.689+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0005470 con 0x7f9ed0071e80 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.689+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed0161ca0 con 0x7f9ed0071e80 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.689+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0012870 con 0x7f9ed0071e80 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.689+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed0162260 con 0x7f9ed0071e80 2024-04-26T21:46:42.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.690+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed00693c0 con 0x7f9ed0071e80 2024-04-26T21:46:42.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.691+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec0012a10 con 0x7f9ed0071e80 2024-04-26T21:46:42.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.692+0000 7f9ec7e00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.692+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9ec008b880 con 0x7f9ed0071e80 2024-04-26T21:46:42.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.692+0000 7f9ecf400640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.693+0000 7f9ecf400640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000c00 tx=0x7f9ebc0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.697+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ec0059c20 con 0x7f9ed0071e80 2024-04-26T21:46:42.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.894+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ed0155a60 con 0x7f9ed0071e80 2024-04-26T21:46:42.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.895+0000 7f9ec7e00640 1 -- 172.21.15.106:0/1390088953 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f9ec0059520 con 0x7f9ed0071e80 2024-04-26T21:46:42.895 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:42.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.897+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 msgr2=0x7f9eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.897+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000c00 tx=0x7f9ebc0079c0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 msgr2=0x7f9ed00cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c480 tx=0x7f9ec000c940 comp rx=0 tx=0).stop 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 shutdown_connections 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ed00b4ed0 0x7f9ed0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed0072970 0x7f9ed00cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 --2- 172.21.15.106:0/1390088953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed0071e80 0x7f9ed00cf2b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 >> 172.21.15.106:0/1390088953 conn(0x7f9ed00b2900 msgr2=0x7f9ed0072260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 shutdown_connections 2024-04-26T21:46:42.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:42.898+0000 7f9ed5a00640 1 -- 172.21.15.106:0/1390088953 wait complete. 2024-04-26T21:46:42.916 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.120+0000 7f00f0600640 1 Processor -- start 2024-04-26T21:46:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.121+0000 7f00f0600640 1 -- start start 2024-04-26T21:46:43.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.122+0000 7f00f0600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f00e80b5ca0 0x7f00e80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.122+0000 7f00f0600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80af800 con 0x7f00e815d100 2024-04-26T21:46:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.122+0000 7f00f0600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80ae340 con 0x7f00e806ce80 2024-04-26T21:46:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.122+0000 7f00f0600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f00e80afaa0 con 0x7f00e80b5ca0 2024-04-26T21:46:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00edc00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f00e815d100 0x7f00e806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48568/0 (socket says 172.21.15.106:48568) 2024-04-26T21:46:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00edc00640 1 -- 172.21.15.106:0/2796447609 learned_addr learned my addr 172.21.15.106:0/2796447609 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00ee600640 1 --2- 172.21.15.106:0/2796447609 >> v2:172.21.15.204:3300/0 conn(0x7f00e80b5ca0 0x7f00e80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996666213 0 0) 0x7f00e80af800 con 0x7f00e815d100 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d0003520 con 0x7f00e815d100 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 975492241 0 0) 0x7f00d0003520 con 0x7f00e815d100 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 >> v2:172.21.15.204:3300/0 conn(0x7f00e80b5ca0 msgr2=0x7f00e80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 --2- 172.21.15.106:0/2796447609 >> v2:172.21.15.204:3300/0 conn(0x7f00e80b5ca0 0x7f00e80b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 >> v1:172.21.15.204:6789/0 conn(0x7f00e806ce80 legacy=0x7f00e80b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80afd80 con 0x7f00e815d100 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f00dc001670 con 0x7f00e815d100 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 >> v1:172.21.15.106:6789/0 conn(0x7f00e815d100 legacy=0x7f00e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.123+0000 7f00e7e00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00e7e00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d0004800 0x7f00d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00e7e00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d00051c0 0x7f00d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80afaa0 con 0x7f00d00039b0 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00d000c160 con 0x7f00d00051c0 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00d000c360 con 0x7f00d0004800 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00ee600640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d00051c0 0x7f00d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00ed200640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d0004800 0x7f00d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 -- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d0004800 msgr2=0x7f00d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d0004800 0x7f00d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 -- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d00051c0 msgr2=0x7f00d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d00051c0 0x7f00d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00edc00640 1 -- 172.21.15.106:0/2796447609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d0003520 con 0x7f00d00039b0 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.124+0000 7f00ee600640 1 -- 172.21.15.106:0/2796447609 reap_dead start 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.125+0000 7f00edc00640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00d0003dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c240 tx=0x7f00d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273eefba8cde66f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007d10 con 0x7f00d00039b0 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0058d0 con 0x7f00d00039b0 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00e7e00640 1 -- 172.21.15.106:0/2796447609 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc005d20 con 0x7f00d00039b0 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 -- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 msgr2=0x7f00d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00d0003dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c240 tx=0x7f00d0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 -- 172.21.15.106:0/2796447609 shutdown_connections 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 --2- 172.21.15.106:0/2796447609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00d0003dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 -- 172.21.15.106:0/2796447609 >> 172.21.15.106:0/2796447609 conn(0x7f00e80b2d50 msgr2=0x7f00e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.126+0000 7f00f0600640 1 -- 172.21.15.106:0/2796447609 shutdown_connections 2024-04-26T21:46:43.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.127+0000 7f00f0600640 1 -- 172.21.15.106:0/2796447609 wait complete. 2024-04-26T21:46:43.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.128+0000 7f00f0600640 1 Processor -- start 2024-04-26T21:46:43.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.128+0000 7f00f0600640 1 -- start start 2024-04-26T21:46:43.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00e8156c00 0x7f00e815f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00e8157080 0x7f00e815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00e809ee50 con 0x7f00d00039b0 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80b0340 con 0x7f00e8156c00 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.129+0000 7f00f0600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00e8160b70 con 0x7f00e8157080 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57890/0 (socket says 172.21.15.106:57890) 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 -- 172.21.15.106:0/2862937881 learned_addr learned my addr 172.21.15.106:0/2862937881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 -- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00e8157080 msgr2=0x7f00e815fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00e8157080 0x7f00e815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 -- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00e8156c00 msgr2=0x7f00e815f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00e8156c00 0x7f00e815f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 -- 172.21.15.106:0/2862937881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e815e0f0 con 0x7f00d00039b0 2024-04-26T21:46:43.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.130+0000 7f00edc00640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02adf0 tx=0x7f00e80ae510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.131+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc038040 con 0x7f00d00039b0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.131+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc042070 con 0x7f00d00039b0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.131+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007d10 con 0x7f00d00039b0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.131+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e809c650 con 0x7f00d00039b0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.131+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e809cc10 con 0x7f00d00039b0 2024-04-26T21:46:43.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.132+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e80ae580 con 0x7f00d00039b0 2024-04-26T21:46:43.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.133+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00dc004be0 con 0x7f00d00039b0 2024-04-26T21:46:43.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.134+0000 7f00e5600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 0x7f00cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.134+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f00dc0abd70 con 0x7f00d00039b0 2024-04-26T21:46:43.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.134+0000 7f00ed200640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 0x7f00cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.135+0000 7f00ed200640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 0x7f00cc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00e809c390 tx=0x7f00d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.142+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00e80ae580 con 0x7f00d00039b0 2024-04-26T21:46:43.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.340+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00e8069650 con 0x7f00d00039b0 2024-04-26T21:46:43.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.341+0000 7f00e5600640 1 -- 172.21.15.106:0/2862937881 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f00dc07a1a0 con 0x7f00d00039b0 2024-04-26T21:46:43.341 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.343+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 msgr2=0x7f00cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.343+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 0x7f00cc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00e809c390 tx=0x7f00d8009040 comp rx=0 tx=0).stop 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 msgr2=0x7f00e81534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02adf0 tx=0x7f00e80ae510 comp rx=0 tx=0).stop 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 shutdown_connections 2024-04-26T21:46:43.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f00cc06c8f0 0x7f00cc06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00e8157080 0x7f00e815fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00e8156c00 0x7f00e815f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 --2- 172.21.15.106:0/2862937881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00039b0 0x7f00e81534d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 >> 172.21.15.106:0/2862937881 conn(0x7f00e80b2d50 msgr2=0x7f00e8157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 shutdown_connections 2024-04-26T21:46:43.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.344+0000 7f00f0600640 1 -- 172.21.15.106:0/2862937881 wait complete. 2024-04-26T21:46:43.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:43.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.556+0000 7fa48d800640 1 Processor -- start 2024-04-26T21:46:43.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.556+0000 7fa48d800640 1 -- start start 2024-04-26T21:46:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.557+0000 7fa48d800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa4880d1470 0x7fa488167da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.557+0000 7fa48d800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa488160b20 con 0x7fa4881467a0 2024-04-26T21:46:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.557+0000 7fa48d800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa488160e00 con 0x7fa4880b5ca0 2024-04-26T21:46:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.557+0000 7fa48d800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa488146b80 con 0x7fa4880d1470 2024-04-26T21:46:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa487e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa4881467a0 0x7fa48815dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48584/0 (socket says 172.21.15.106:48584) 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa487e00640 1 -- 172.21.15.106:0/3151738488 learned_addr learned my addr 172.21.15.106:0/3151738488 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa48ce00640 1 --2- 172.21.15.106:0/3151738488 >> v2:172.21.15.106:3300/0 conn(0x7fa4880d1470 0x7fa488167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa48ce00640 1 -- 172.21.15.106:0/3151738488 >> v1:172.21.15.204:6789/0 conn(0x7fa4880b5ca0 legacy=0x7fa4880d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa48ce00640 1 -- 172.21.15.106:0/3151738488 >> v1:172.21.15.106:6789/0 conn(0x7fa4881467a0 legacy=0x7fa48815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa486a00640 1 -- 172.21.15.106:0/3151738488 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612984682 0 0) 0x7fa488160b20 con 0x7fa4881467a0 2024-04-26T21:46:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.558+0000 7fa48ce00640 1 -- 172.21.15.106:0/3151738488 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4881610e0 con 0x7fa4880d1470 2024-04-26T21:46:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.559+0000 7fa48ce00640 1 --2- 172.21.15.106:0/3151738488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa488167da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa488146d00 tx=0x7fa48002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecb2bb01e16b63b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa486a00640 1 -- 172.21.15.106:0/3151738488 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480007740 con 0x7fa4880d1470 2024-04-26T21:46:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa486a00640 1 -- 172.21.15.106:0/3151738488 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4800078e0 con 0x7fa4880d1470 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa486a00640 1 -- 172.21.15.106:0/3151738488 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480007cb0 con 0x7fa4880d1470 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 -- 172.21.15.106:0/3151738488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 msgr2=0x7fa488167da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 --2- 172.21.15.106:0/3151738488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa488167da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa488146d00 tx=0x7fa48002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 -- 172.21.15.106:0/3151738488 shutdown_connections 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 --2- 172.21.15.106:0/3151738488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa488167da0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 -- 172.21.15.106:0/3151738488 >> 172.21.15.106:0/3151738488 conn(0x7fa48806c530 msgr2=0x7fa48806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 -- 172.21.15.106:0/3151738488 shutdown_connections 2024-04-26T21:46:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.560+0000 7fa48d800640 1 -- 172.21.15.106:0/3151738488 wait complete. 2024-04-26T21:46:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.562+0000 7fa48d800640 1 Processor -- start 2024-04-26T21:46:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.562+0000 7fa48d800640 1 -- start start 2024-04-26T21:46:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 0x7fa48815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4881467a0 0x7fa4880afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4880b5100 con 0x7fa4880d1470 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa488161730 con 0x7fa4880b5ca0 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa48d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa4880b5280 con 0x7fa4881467a0 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa487400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa487400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57894/0 (socket says 172.21.15.106:57894) 2024-04-26T21:46:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.563+0000 7fa487400640 1 -- 172.21.15.106:0/3077152732 learned_addr learned my addr 172.21.15.106:0/3077152732 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487e00640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 0x7fa48815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 -- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4881467a0 msgr2=0x7fa4880afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4881467a0 0x7fa4880afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 -- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 msgr2=0x7fa48815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 0x7fa48815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 -- 172.21.15.106:0/3077152732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4881f6400 con 0x7fa4880d1470 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487e00640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 0x7fa48815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa487400640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa47400c430 tx=0x7fa47400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474011790 con 0x7fa4880d1470 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.564+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa474017070 con 0x7fa4880d1470 2024-04-26T21:46:43.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.565+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474011bb0 con 0x7fa4880d1470 2024-04-26T21:46:43.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.565+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4881f66e0 con 0x7fa4880d1470 2024-04-26T21:46:43.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.565+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4881f6ca0 con 0x7fa4880d1470 2024-04-26T21:46:43.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.566+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4880693c0 con 0x7fa4880d1470 2024-04-26T21:46:43.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.567+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa47401c460 con 0x7fa4880d1470 2024-04-26T21:46:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.568+0000 7fa47fe00640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.568+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa47408ba70 con 0x7fa4880d1470 2024-04-26T21:46:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.568+0000 7fa487e00640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.569+0000 7fa487e00640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa478000f60 tx=0x7fa478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.572+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa474059e10 con 0x7fa4880d1470 2024-04-26T21:46:43.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.760+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa48806f7a0 con 0x7fa4880d1470 2024-04-26T21:46:43.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.761+0000 7fa47fe00640 1 -- 172.21.15.106:0/3077152732 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa474059710 con 0x7fa4880d1470 2024-04-26T21:46:43.761 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:43.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 msgr2=0x7fa46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa478000f60 tx=0x7fa478009040 comp rx=0 tx=0).stop 2024-04-26T21:46:43.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 msgr2=0x7fa4880abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa47400c430 tx=0x7fa47400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 shutdown_connections 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4881467a0 0x7fa4880afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4880d1470 0x7fa4880abb50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 --2- 172.21.15.106:0/3077152732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4880b5ca0 0x7fa48815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.763+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 >> 172.21.15.106:0/3077152732 conn(0x7fa48806c530 msgr2=0x7fa4880b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.764+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 shutdown_connections 2024-04-26T21:46:43.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.764+0000 7fa48d800640 1 -- 172.21.15.106:0/3077152732 wait complete. 2024-04-26T21:46:43.782 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:43.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.979+0000 7faf53800640 1 Processor -- start 2024-04-26T21:46:43.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.980+0000 7faf53800640 1 -- start start 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf53800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faf4c15ef60 0x7faf4c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf53800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4c0b5d30 con 0x7faf4c0d34a0 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf53800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4c0b6010 con 0x7faf4c0d5d90 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf53800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7faf4c0b62f0 con 0x7faf4c15ef60 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf50e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faf4c15ef60 0x7faf4c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf50e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faf4c15ef60 0x7faf4c15f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57910/0 (socket says 172.21.15.106:57910) 2024-04-26T21:46:43.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf51800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7faf4c0d34a0 0x7faf4c0d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48586/0 (socket says 172.21.15.106:48586) 2024-04-26T21:46:43.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.981+0000 7faf50e00640 1 -- 172.21.15.106:0/385042417 learned_addr learned my addr 172.21.15.106:0/385042417 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:43.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.982+0000 7faf50e00640 1 -- 172.21.15.106:0/385042417 >> v1:172.21.15.204:6789/0 conn(0x7faf4c0d5d90 legacy=0x7faf4c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.982+0000 7faf50e00640 1 -- 172.21.15.106:0/385042417 >> v1:172.21.15.106:6789/0 conn(0x7faf4c0d34a0 legacy=0x7faf4c0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.982+0000 7faf50e00640 1 -- 172.21.15.106:0/385042417 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c0b65d0 con 0x7faf4c15ef60 2024-04-26T21:46:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.983+0000 7faf50e00640 1 --2- 172.21.15.106:0/385042417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c15f340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0098a0 tx=0x7faf3c02a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff5ba345b52094b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.983+0000 7faf4be00640 1 -- 172.21.15.106:0/385042417 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c037070 con 0x7faf4c15ef60 2024-04-26T21:46:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.983+0000 7faf4be00640 1 -- 172.21.15.106:0/385042417 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3c032640 con 0x7faf4c15ef60 2024-04-26T21:46:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.983+0000 7faf4be00640 1 -- 172.21.15.106:0/385042417 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c032a10 con 0x7faf4c15ef60 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 -- 172.21.15.106:0/385042417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 msgr2=0x7faf4c15f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 --2- 172.21.15.106:0/385042417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c15f340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0098a0 tx=0x7faf3c02a370 comp rx=0 tx=0).stop 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 -- 172.21.15.106:0/385042417 shutdown_connections 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 --2- 172.21.15.106:0/385042417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c15f340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 -- 172.21.15.106:0/385042417 >> 172.21.15.106:0/385042417 conn(0x7faf4c09e3b0 msgr2=0x7faf4c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 -- 172.21.15.106:0/385042417 shutdown_connections 2024-04-26T21:46:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.984+0000 7faf53800640 1 -- 172.21.15.106:0/385042417 wait complete. 2024-04-26T21:46:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.986+0000 7faf53800640 1 Processor -- start 2024-04-26T21:46:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.986+0000 7faf53800640 1 -- start start 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 0x7faf4c15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf4c0d5d90 0x7faf4c1f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c0d3ab0 con 0x7faf4c15ef60 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c0d3930 con 0x7faf4c0d34a0 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf53800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c0d3c30 con 0x7faf4c0d5d90 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf51800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf51800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57914/0 (socket says 172.21.15.106:57914) 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf51800640 1 -- 172.21.15.106:0/2195603187 learned_addr learned my addr 172.21.15.106:0/2195603187 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.987+0000 7faf50e00640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 0x7faf4c15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 -- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf4c0d5d90 msgr2=0x7faf4c1f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf43e00640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf4c0d5d90 0x7faf4c1f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf4c0d5d90 0x7faf4c1f6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 -- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 msgr2=0x7faf4c15b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 0x7faf4c15b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 -- 172.21.15.106:0/2195603187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c1f6d90 con 0x7faf4c15ef60 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf50e00640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 0x7faf4c15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf51800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c730 tx=0x7faf4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44017070 con 0x7faf4c15ef60 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4400ce40 con 0x7faf4c15ef60 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c1f6440 con 0x7faf4c15ef60 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400a840 con 0x7faf4c15ef60 2024-04-26T21:46:43.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.988+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c0cb430 con 0x7faf4c15ef60 2024-04-26T21:46:43.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.989+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c069540 con 0x7faf4c15ef60 2024-04-26T21:46:43.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.991+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf4401a070 con 0x7faf4c15ef60 2024-04-26T21:46:43.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.991+0000 7faf49600640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.992+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7faf4408b040 con 0x7faf4c15ef60 2024-04-26T21:46:43.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.992+0000 7faf50e00640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.992+0000 7faf50e00640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:43.996+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf44058da0 con 0x7faf4c15ef60 2024-04-26T21:46:44.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.210+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf4c08d090 con 0x7faf4c15ef60 2024-04-26T21:46:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.211+0000 7faf49600640 1 -- 172.21.15.106:0/2195603187 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7faf4405d0a0 con 0x7faf4c15ef60 2024-04-26T21:46:44.211 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:44.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.213+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 msgr2=0x7faf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.213+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.213+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 msgr2=0x7faf4c0c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.213+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c730 tx=0x7faf4400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.213+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 shutdown_connections 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf4c15ef60 0x7faf4c0c99e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf4c0d5d90 0x7faf4c1f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 --2- 172.21.15.106:0/2195603187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf4c0d34a0 0x7faf4c15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 >> 172.21.15.106:0/2195603187 conn(0x7faf4c09e3b0 msgr2=0x7faf4c1ebce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 shutdown_connections 2024-04-26T21:46:44.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.214+0000 7faf53800640 1 -- 172.21.15.106:0/2195603187 wait complete. 2024-04-26T21:46:44.232 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-26T21:46:44.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:44.233 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:44.318 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:46:44.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:46:44.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:44.319 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:44.378 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:44.378 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:44.434 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:44.434 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:44.486 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:44.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.693+0000 7efc23200640 1 Processor -- start 2024-04-26T21:46:44.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.694+0000 7efc23200640 1 -- start start 2024-04-26T21:46:44.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.695+0000 7efc23200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc240d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.695+0000 7efc23200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efc240d1f60 0x7efc240d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.695+0000 7efc23200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240d05b0 con 0x7efc2415f3a0 2024-04-26T21:46:44.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.695+0000 7efc23200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efc240d57c0 con 0x7efc240d1f60 2024-04-26T21:46:44.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.695+0000 7efc23200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc240d0b90 con 0x7efc240b2b30 2024-04-26T21:46:44.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc21e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efc2415f3a0 0x7efc240d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48602/0 (socket says 172.21.15.106:48602) 2024-04-26T21:46:44.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc21e00640 1 -- 172.21.15.106:0/4253869832 learned_addr learned my addr 172.21.15.106:0/4253869832 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc22800640 1 --2- 172.21.15.106:0/4253869832 >> v2:172.21.15.204:3300/0 conn(0x7efc240d1f60 0x7efc240d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc21400640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc240d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106629418 0 0) 0x7efc240d05b0 con 0x7efc2415f3a0 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.696+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc08003520 con 0x7efc2415f3a0 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1179737994 0 0) 0x7efc08003520 con 0x7efc2415f3a0 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 msgr2=0x7efc240d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc240d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 >> v2:172.21.15.204:3300/0 conn(0x7efc240d1f60 msgr2=0x7efc240d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 --2- 172.21.15.106:0/4253869832 >> v2:172.21.15.204:3300/0 conn(0x7efc240d1f60 0x7efc240d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240d5a10 con 0x7efc2415f3a0 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7efc14001670 con 0x7efc2415f3a0 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 >> v1:172.21.15.106:6789/0 conn(0x7efc2415f3a0 legacy=0x7efc240d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc080039b0 0x7efc08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc08004800 0x7efc08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d57c0 con 0x7efc080039b0 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d0b90 con 0x7efc08004800 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc0800b9c0 con 0x7efc240b2b30 2024-04-26T21:46:44.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.697+0000 7efc21e00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc080039b0 0x7efc08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21400640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc08004800 0x7efc08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc22800640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21e00640 1 -- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 msgr2=0x7efc080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:44.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21e00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc240b2b30 0x7efc080051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21e00640 1 -- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc08004800 msgr2=0x7efc08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21e00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc08004800 0x7efc08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21e00640 1 -- 172.21.15.106:0/4253869832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc08003520 con 0x7efc080039b0 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.698+0000 7efc21400640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc08004800 0x7efc08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.699+0000 7efc21e00640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc080039b0 0x7efc08003dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c190 tx=0x7efc08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a386ce0b40861a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.699+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007c80 con 0x7efc080039b0 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.699+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1402e070 con 0x7efc080039b0 2024-04-26T21:46:44.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.699+0000 7efc20a00640 1 -- 172.21.15.106:0/4253869832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14005d30 con 0x7efc080039b0 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 -- 172.21.15.106:0/4253869832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc080039b0 msgr2=0x7efc08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 --2- 172.21.15.106:0/4253869832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc080039b0 0x7efc08003dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c190 tx=0x7efc08005830 comp rx=0 tx=0).stop 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc22800640 1 -- 172.21.15.106:0/4253869832 reap_dead start 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 -- 172.21.15.106:0/4253869832 shutdown_connections 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 -- 172.21.15.106:0/4253869832 >> 172.21.15.106:0/4253869832 conn(0x7efc2409e1c0 msgr2=0x7efc240d4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 -- 172.21.15.106:0/4253869832 shutdown_connections 2024-04-26T21:46:44.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.700+0000 7efc23200640 1 -- 172.21.15.106:0/4253869832 wait complete. 2024-04-26T21:46:44.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.702+0000 7efc23200640 1 Processor -- start 2024-04-26T21:46:44.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.702+0000 7efc23200640 1 -- start start 2024-04-26T21:46:44.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc080039b0 0x7efc240b9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 0x7efc24153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc240b3cf0 con 0x7efc240ba230 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d5cf0 con 0x7efc241532e0 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.703+0000 7efc23200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc240b3e70 con 0x7efc080039b0 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57930/0 (socket says 172.21.15.106:57930) 2024-04-26T21:46:44.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 -- 172.21.15.106:0/1444039563 learned_addr learned my addr 172.21.15.106:0/1444039563 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21e00640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc080039b0 0x7efc240b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc22800640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 0x7efc24153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 -- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc080039b0 msgr2=0x7efc240b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc080039b0 0x7efc240b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 -- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 msgr2=0x7efc24153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 0x7efc24153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc21400640 1 -- 172.21.15.106:0/1444039563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24156100 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.704+0000 7efc22800640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 0x7efc24153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc21400640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc100108c0 tx=0x7efc10010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1000f040 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1000eb00 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241572d0 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1001c400 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.705+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc240ba6b0 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.706+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24157c00 con 0x7efc240ba230 2024-04-26T21:46:44.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.707+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc1001c5a0 con 0x7efc240ba230 2024-04-26T21:46:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.709+0000 7efc1a000640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.709+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efc1008c160 con 0x7efc240ba230 2024-04-26T21:46:44.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.710+0000 7efc21e00640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:44.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.710+0000 7efc21e00640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc14032000 tx=0x7efc14007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:44.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.714+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc10018030 con 0x7efc240ba230 2024-04-26T21:46:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.919+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc241570e0 con 0x7efc240ba230 2024-04-26T21:46:44.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.922+0000 7efc1a000640 1 -- 172.21.15.106:0/1444039563 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7efc1005a5a0 con 0x7efc240ba230 2024-04-26T21:46:44.923 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:44.923 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:44.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.924+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 msgr2=0x7efc0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.924+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc14032000 tx=0x7efc14007000 comp rx=0 tx=0).stop 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 msgr2=0x7efc24152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc100108c0 tx=0x7efc10010d80 comp rx=0 tx=0).stop 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 shutdown_connections 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc241532e0 0x7efc24153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc240ba230 0x7efc24152da0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 --2- 172.21.15.106:0/1444039563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc080039b0 0x7efc240b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 >> 172.21.15.106:0/1444039563 conn(0x7efc2409e1c0 msgr2=0x7efc240d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:44.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 shutdown_connections 2024-04-26T21:46:44.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:44.925+0000 7efc23200640 1 -- 172.21.15.106:0/1444039563 wait complete. 2024-04-26T21:46:44.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:44.945 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:45.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.147+0000 7f555a600640 1 Processor -- start 2024-04-26T21:46:45.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.147+0000 7f555a600640 1 -- start start 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f555a600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5554161140 0x7f555406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f555a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 0x7f55541466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f555a600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55540b4d20 con 0x7f5554160d60 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f555a600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5554146c30 con 0x7f5554161140 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f555a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f55540b5000 con 0x7f5554146280 2024-04-26T21:46:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f5553e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5554161140 0x7f555406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f5559200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5554160d60 0x7f55540b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48606/0 (socket says 172.21.15.106:48606) 2024-04-26T21:46:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.149+0000 7f5559200640 1 -- 172.21.15.106:0/3056286890 learned_addr learned my addr 172.21.15.106:0/3056286890 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5559c00640 1 --2- 172.21.15.106:0/3056286890 >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 0x7f55541466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553400640 1 -- 172.21.15.106:0/3056286890 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124709517 0 0) 0x7f55540b4d20 con 0x7f5554160d60 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553400640 1 -- 172.21.15.106:0/3056286890 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5538003520 con 0x7f5554160d60 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553e00640 1 -- 172.21.15.106:0/3056286890 >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 msgr2=0x7f55541466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553e00640 1 --2- 172.21.15.106:0/3056286890 >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 0x7f55541466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553e00640 1 -- 172.21.15.106:0/3056286890 >> v1:172.21.15.106:6789/0 conn(0x7f5554160d60 legacy=0x7f55540b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5553e00640 1 -- 172.21.15.106:0/3056286890 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55540b52e0 con 0x7f5554161140 2024-04-26T21:46:45.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.150+0000 7f5559c00640 1 --2- 172.21.15.106:0/3056286890 >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 0x7f55541466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.151+0000 7f5553e00640 1 --2- 172.21.15.106:0/3056286890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f555406dee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f553c0097a0 tx=0x7f553c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb706dea96c288ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.151+0000 7f5553400640 1 -- 172.21.15.106:0/3056286890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c037070 con 0x7f5554161140 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.151+0000 7f5553400640 1 -- 172.21.15.106:0/3056286890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c032440 con 0x7f5554161140 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.151+0000 7f5553400640 1 -- 172.21.15.106:0/3056286890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0327f0 con 0x7f5554161140 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 -- 172.21.15.106:0/3056286890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 msgr2=0x7f555406dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 --2- 172.21.15.106:0/3056286890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f555406dee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f553c0097a0 tx=0x7f553c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 -- 172.21.15.106:0/3056286890 shutdown_connections 2024-04-26T21:46:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 --2- 172.21.15.106:0/3056286890 >> v2:172.21.15.204:3300/0 conn(0x7f5554146280 0x7f55541466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 --2- 172.21.15.106:0/3056286890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f555406dee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 -- 172.21.15.106:0/3056286890 >> 172.21.15.106:0/3056286890 conn(0x7f55540b2900 msgr2=0x7f55540b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 -- 172.21.15.106:0/3056286890 shutdown_connections 2024-04-26T21:46:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.152+0000 7f555a600640 1 -- 172.21.15.106:0/3056286890 wait complete. 2024-04-26T21:46:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.154+0000 7f555a600640 1 Processor -- start 2024-04-26T21:46:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.154+0000 7f555a600640 1 -- start start 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5554146280 0x7f55541596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 0x7f55540cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f555406e420 con 0x7f5554161140 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55540cf6f0 con 0x7f5554146280 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f555a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f555406e5a0 con 0x7f5554160d60 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.155+0000 7f5559c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57954/0 (socket says 172.21.15.106:57954) 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 -- 172.21.15.106:0/1851544120 learned_addr learned my addr 172.21.15.106:0/1851544120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559200640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5554146280 0x7f55541596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5553e00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 0x7f55540cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 -- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 msgr2=0x7f55540cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 0x7f55540cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 -- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5554146280 msgr2=0x7f55541596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5554146280 0x7f55541596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 -- 172.21.15.106:0/1851544120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55540d1150 con 0x7f5554161140 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5553e00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 0x7f55540cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.156+0000 7f5559c00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f554400c610 tx=0x7f554400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.157+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544015070 con 0x7f5554161140 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.157+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5544011800 con 0x7f5554161140 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.157+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544011bf0 con 0x7f5554161140 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.157+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55540ca020 con 0x7f5554161140 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.157+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55540ca5e0 con 0x7f5554161140 2024-04-26T21:46:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.158+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55540ceb90 con 0x7f5554161140 2024-04-26T21:46:45.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.159+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5544018070 con 0x7f5554161140 2024-04-26T21:46:45.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.160+0000 7f5550c00640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 0x7f553006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.160+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f554408bab0 con 0x7f5554161140 2024-04-26T21:46:45.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.160+0000 7f5559200640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 0x7f553006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.161+0000 7f5559200640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 0x7f553006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f554c005c00 tx=0x7f554c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.165+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55540ceb90 con 0x7f5554161140 2024-04-26T21:46:45.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:45.205 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:45.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.372+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55540693c0 con 0x7f5554161140 2024-04-26T21:46:45.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.373+0000 7f5550c00640 1 -- 172.21.15.106:0/1851544120 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f5544059e50 con 0x7f5554161140 2024-04-26T21:46:45.373 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:45.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 msgr2=0x7f553006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 0x7f553006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f554c005c00 tx=0x7f554c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:45.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 msgr2=0x7f55540cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f554400c610 tx=0x7f554400cad0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 shutdown_connections 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5554161140 0x7f55540cce60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f553006c8f0 0x7f553006edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5554160d60 0x7f55540cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 --2- 172.21.15.106:0/1851544120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5554146280 0x7f55541596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.375+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 >> 172.21.15.106:0/1851544120 conn(0x7f55540b2900 msgr2=0x7f55540d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:45.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.376+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 shutdown_connections 2024-04-26T21:46:45.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.376+0000 7f555a600640 1 -- 172.21.15.106:0/1851544120 wait complete. 2024-04-26T21:46:45.395 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.588+0000 7f64b4600640 1 Processor -- start 2024-04-26T21:46:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.588+0000 7f64b4600640 1 -- start start 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b4600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f64ac0d19c0 0x7f64ac0d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b4600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64ac0b6170 0x7f64ac0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b4600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ac0d3830 con 0x7f64ac0d1da0 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b4600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0d40d0 con 0x7f64ac0b6170 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b4600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0d3b10 con 0x7f64ac0d19c0 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64ac0b6170 0x7f64ac0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64ac0b6170 0x7f64ac0c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57968/0 (socket says 172.21.15.106:57968) 2024-04-26T21:46:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1200640 1 -- 172.21.15.106:0/3781070942 learned_addr learned my addr 172.21.15.106:0/3781070942 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1c00640 1 --2- 172.21.15.106:0/3781070942 >> v2:172.21.15.204:3300/0 conn(0x7f64ac0d19c0 0x7f64ac0d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1200640 1 -- 172.21.15.106:0/3781070942 >> v2:172.21.15.204:3300/0 conn(0x7f64ac0d19c0 msgr2=0x7f64ac0d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.590+0000 7f64b1200640 1 --2- 172.21.15.106:0/3781070942 >> v2:172.21.15.204:3300/0 conn(0x7f64ac0d19c0 0x7f64ac0d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.591+0000 7f64b1200640 1 -- 172.21.15.106:0/3781070942 >> v1:172.21.15.204:6789/0 conn(0x7f64ac0d1da0 legacy=0x7f64ac162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.591+0000 7f64b1200640 1 -- 172.21.15.106:0/3781070942 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0d3df0 con 0x7f64ac0b6170 2024-04-26T21:46:45.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.592+0000 7f64b1200640 1 --2- 172.21.15.106:0/3781070942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0b6170 0x7f64ac0c8c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06ca80 tx=0x7f649802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93846aa3e6db2244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.592+0000 7f64abe00640 1 -- 172.21.15.106:0/3781070942 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498037070 con 0x7f64ac0b6170 2024-04-26T21:46:45.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64abe00640 1 -- 172.21.15.106:0/3781070942 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498032440 con 0x7f64ac0b6170 2024-04-26T21:46:45.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64abe00640 1 -- 172.21.15.106:0/3781070942 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498032810 con 0x7f64ac0b6170 2024-04-26T21:46:45.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 -- 172.21.15.106:0/3781070942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0b6170 msgr2=0x7f64ac0c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 --2- 172.21.15.106:0/3781070942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0b6170 0x7f64ac0c8c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06ca80 tx=0x7f649802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 -- 172.21.15.106:0/3781070942 shutdown_connections 2024-04-26T21:46:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 --2- 172.21.15.106:0/3781070942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0b6170 0x7f64ac0c8c70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 --2- 172.21.15.106:0/3781070942 >> v2:172.21.15.204:3300/0 conn(0x7f64ac0d19c0 0x7f64ac0d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 -- 172.21.15.106:0/3781070942 >> 172.21.15.106:0/3781070942 conn(0x7f64ac09e460 msgr2=0x7f64ac0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.593+0000 7f64b4600640 1 -- 172.21.15.106:0/3781070942 shutdown_connections 2024-04-26T21:46:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.594+0000 7f64b4600640 1 -- 172.21.15.106:0/3781070942 wait complete. 2024-04-26T21:46:45.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.596+0000 7f64b4600640 1 Processor -- start 2024-04-26T21:46:45.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.596+0000 7f64b4600640 1 -- start start 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 0x7f64ac157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64ac0d1da0 0x7f64ac15e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0cfc30 con 0x7f64ac0d19c0 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0c91b0 con 0x7f64ac0d1da0 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac162d50 con 0x7f64ac0b6170 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57982/0 (socket says 172.21.15.106:57982) 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.598+0000 7f64b1200640 1 -- 172.21.15.106:0/702968502 learned_addr learned my addr 172.21.15.106:0/702968502 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:45.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 -- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 msgr2=0x7f64ac157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1c00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 0x7f64ac157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b2600640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64ac0d1da0 0x7f64ac15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 0x7f64ac157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 -- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64ac0d1da0 msgr2=0x7f64ac15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64ac0d1da0 0x7f64ac15e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 -- 172.21.15.106:0/702968502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0cdf40 con 0x7f64ac0d19c0 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1c00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 0x7f64ac157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:45.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b1200640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6498000c00 tx=0x7f64980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498041020 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498030820 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498037040 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.599+0000 7f64b4600640 1 -- 172.21.15.106:0/702968502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac0ce220 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.600+0000 7f64b4600640 1 -- 172.21.15.106:0/702968502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac0ceac0 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.601+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac0ce610 con 0x7f64ac0d19c0 2024-04-26T21:46:45.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.602+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6498044020 con 0x7f64ac0d19c0 2024-04-26T21:46:45.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.603+0000 7f64a9600640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 0x7f648c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:45.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.603+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f64980ac520 con 0x7f64ac0d19c0 2024-04-26T21:46:45.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.603+0000 7f64b1c00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 0x7f648c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:45.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.604+0000 7f64b1c00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 0x7f648c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64a0008890 tx=0x7f64a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:45.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.607+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f649804b030 con 0x7f64ac0d19c0 2024-04-26T21:46:45.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.828+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64ac0ce3f0 con 0x7f64ac0d19c0 2024-04-26T21:46:45.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.829+0000 7f64a9600640 1 -- 172.21.15.106:0/702968502 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f649807a830 con 0x7f64ac0d19c0 2024-04-26T21:46:45.829 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 msgr2=0x7f648c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 0x7f648c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64a0008890 tx=0x7f64a0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 msgr2=0x7f64ac0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6498000c00 tx=0x7f64980046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 shutdown_connections 2024-04-26T21:46:45.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64ac0d1da0 0x7f64ac15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64ac0d19c0 0x7f64ac0cd4a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f648c06c920 0x7f648c06ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 --2- 172.21.15.106:0/702968502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64ac0b6170 0x7f64ac157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 >> 172.21.15.106:0/702968502 conn(0x7f64ac09e460 msgr2=0x7f64ac0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 shutdown_connections 2024-04-26T21:46:45.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:45.832+0000 7f649ea00640 1 -- 172.21.15.106:0/702968502 wait complete. 2024-04-26T21:46:45.850 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:46.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.039+0000 7f8b2b000640 1 Processor -- start 2024-04-26T21:46:46.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.040+0000 7f8b2b000640 1 -- start start 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b2b000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b2415d100 0x7f8b2406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b2b000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b240add80 0x7f8b2406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b2b000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b240af800 con 0x7f8b240b5ca0 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b2b000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8b240ae340 con 0x7f8b2415d100 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b2b000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8b240afaa0 con 0x7f8b240add80 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b23e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b2415d100 0x7f8b2406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b23e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b2415d100 0x7f8b2406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57994/0 (socket says 172.21.15.106:57994) 2024-04-26T21:46:46.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.041+0000 7f8b23e00640 1 -- 172.21.15.106:0/2503799713 learned_addr learned my addr 172.21.15.106:0/2503799713 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:46.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.042+0000 7f8b23e00640 1 -- 172.21.15.106:0/2503799713 >> v2:172.21.15.204:3300/0 conn(0x7f8b240add80 msgr2=0x7f8b2406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.042+0000 7f8b23e00640 1 --2- 172.21.15.106:0/2503799713 >> v2:172.21.15.204:3300/0 conn(0x7f8b240add80 0x7f8b2406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.042+0000 7f8b23e00640 1 -- 172.21.15.106:0/2503799713 >> v1:172.21.15.204:6789/0 conn(0x7f8b240b5ca0 legacy=0x7f8b24072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.042+0000 7f8b23e00640 1 -- 172.21.15.106:0/2503799713 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b240afd80 con 0x7f8b2415d100 2024-04-26T21:46:46.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.043+0000 7f8b23e00640 1 --2- 172.21.15.106:0/2503799713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b2415d100 0x7f8b2406c770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b24073560 tx=0x7f8b1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a739bbf4e1e60a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.043+0000 7f8b23400640 1 -- 172.21.15.106:0/2503799713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14036070 con 0x7f8b2415d100 2024-04-26T21:46:46.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.043+0000 7f8b23400640 1 -- 172.21.15.106:0/2503799713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14031600 con 0x7f8b2415d100 2024-04-26T21:46:46.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.043+0000 7f8b23400640 1 -- 172.21.15.106:0/2503799713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140319d0 con 0x7f8b2415d100 2024-04-26T21:46:46.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 -- 172.21.15.106:0/2503799713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b2415d100 msgr2=0x7f8b2406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2503799713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b2415d100 0x7f8b2406c770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b24073560 tx=0x7f8b1402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 -- 172.21.15.106:0/2503799713 shutdown_connections 2024-04-26T21:46:46.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2503799713 >> v2:172.21.15.204:3300/0 conn(0x7f8b240add80 0x7f8b2406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2503799713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b2415d100 0x7f8b2406c770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 -- 172.21.15.106:0/2503799713 >> 172.21.15.106:0/2503799713 conn(0x7f8b240b2d50 msgr2=0x7f8b240b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:46.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 -- 172.21.15.106:0/2503799713 shutdown_connections 2024-04-26T21:46:46.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.044+0000 7f8b2b000640 1 -- 172.21.15.106:0/2503799713 wait complete. 2024-04-26T21:46:46.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.046+0000 7f8b2b000640 1 Processor -- start 2024-04-26T21:46:46.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.046+0000 7f8b2b000640 1 -- start start 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b240b5ca0 0x7f8b241487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2415d100 0x7f8b24072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2409ee50 con 0x7f8b240add80 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b240b0340 con 0x7f8b2415d100 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b2b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8b24160b70 con 0x7f8b240b5ca0 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.047+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58004/0 (socket says 172.21.15.106:58004) 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 -- 172.21.15.106:0/2881893168 learned_addr learned my addr 172.21.15.106:0/2881893168 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:46.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 -- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b240b5ca0 msgr2=0x7f8b241487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b1b400640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b240b5ca0 0x7f8b241487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b240b5ca0 0x7f8b241487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 -- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2415d100 msgr2=0x7f8b24072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2415d100 0x7f8b24072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 -- 172.21.15.106:0/2881893168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b24148d00 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.048+0000 7f8b23e00640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b14034040 tx=0x7f8b14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.049+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14036050 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.049+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14042070 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.049+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14009510 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.049+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b24155df0 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.049+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b241563b0 con 0x7f8b240add80 2024-04-26T21:46:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.050+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b24069650 con 0x7f8b240add80 2024-04-26T21:46:46.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.051+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b14007c30 con 0x7f8b240add80 2024-04-26T21:46:46.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.052+0000 7f8b20c00640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 0x7f8b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.052+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8b140acb20 con 0x7f8b240add80 2024-04-26T21:46:46.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.052+0000 7f8b1b400640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 0x7f8b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.053+0000 7f8b1b400640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 0x7f8b0006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b241f30c0 tx=0x7f8b0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.060+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b14045360 con 0x7f8b240add80 2024-04-26T21:46:46.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.246+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b2406fce0 con 0x7f8b240add80 2024-04-26T21:46:46.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.247+0000 7f8b20c00640 1 -- 172.21.15.106:0/2881893168 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8b1407af60 con 0x7f8b240add80 2024-04-26T21:46:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:46.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 msgr2=0x7f8b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 0x7f8b0006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b241f30c0 tx=0x7f8b0c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:46.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 msgr2=0x7f8b24156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b14034040 tx=0x7f8b14004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 shutdown_connections 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2415d100 0x7f8b24072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8b0006c8f0 0x7f8b0006edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b240b5ca0 0x7f8b241487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 --2- 172.21.15.106:0/2881893168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b240add80 0x7f8b24156bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 >> 172.21.15.106:0/2881893168 conn(0x7f8b240b2d50 msgr2=0x7f8b2406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 shutdown_connections 2024-04-26T21:46:46.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.250+0000 7f8b2b000640 1 -- 172.21.15.106:0/2881893168 wait complete. 2024-04-26T21:46:46.267 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:46.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.438+0000 7f62aee00640 1 Processor -- start 2024-04-26T21:46:46.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.438+0000 7f62aee00640 1 -- start start 2024-04-26T21:46:46.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.439+0000 7f62aee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 0x7f62a8163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.439+0000 7f62aee00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62a80b82b0 0x7f62a80b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.439+0000 7f62aee00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a80d3f50 con 0x7f62a815f1b0 2024-04-26T21:46:46.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.439+0000 7f62aee00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f62a80d36b0 con 0x7f62a80b82b0 2024-04-26T21:46:46.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.439+0000 7f62aee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62a80d3990 con 0x7f62a80d5990 2024-04-26T21:46:46.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62ada00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f62a815f1b0 0x7f62a8162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48610/0 (socket says 172.21.15.106:48610) 2024-04-26T21:46:46.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62ada00640 1 -- 172.21.15.106:0/2565202543 learned_addr learned my addr 172.21.15.106:0/2565202543 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814814400 0 0) 0x7f62a80d3f50 con 0x7f62a815f1b0 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6290003520 con 0x7f62a815f1b0 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62ad000640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 0x7f62a8163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62ae400640 1 --2- 172.21.15.106:0/2565202543 >> v2:172.21.15.204:3300/0 conn(0x7f62a80b82b0 0x7f62a80b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 306096282 0 0) 0x7f6290003520 con 0x7f62a815f1b0 2024-04-26T21:46:46.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 msgr2=0x7f62a8163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 0x7f62a8163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 >> v2:172.21.15.204:3300/0 conn(0x7f62a80b82b0 msgr2=0x7f62a80b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 --2- 172.21.15.106:0/2565202543 >> v2:172.21.15.204:3300/0 conn(0x7f62a80b82b0 0x7f62a80b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a80d3c70 con 0x7f62a815f1b0 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.440+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f629c001670 con 0x7f62a815f1b0 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 >> v1:172.21.15.106:6789/0 conn(0x7f62a815f1b0 legacy=0x7f62a8162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62900039b0 0x7f6290003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6290004800 0x7f6290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 0x7f62900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80d36b0 con 0x7f62900039b0 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80d3990 con 0x7f6290004800 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f629000bac0 con 0x7f62a80d5990 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62900039b0 0x7f6290003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 -- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 msgr2=0x7f62900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80d5990 0x7f62900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 -- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6290004800 msgr2=0x7f6290004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6290004800 0x7f6290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.441+0000 7f62ada00640 1 -- 172.21.15.106:0/2565202543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6290003520 con 0x7f62900039b0 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.442+0000 7f62ada00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62900039b0 0x7f6290003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c160 tx=0x7f6290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31b12e88085327 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.442+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c007c80 con 0x7f62900039b0 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.442+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629c005ad0 con 0x7f62900039b0 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.442+0000 7f62a7e00640 1 -- 172.21.15.106:0/2565202543 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c007070 con 0x7f62900039b0 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 -- 172.21.15.106:0/2565202543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62900039b0 msgr2=0x7f6290003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 --2- 172.21.15.106:0/2565202543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62900039b0 0x7f6290003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c160 tx=0x7f6290005830 comp rx=0 tx=0).stop 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62ae400640 1 -- 172.21.15.106:0/2565202543 reap_dead start 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 -- 172.21.15.106:0/2565202543 shutdown_connections 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 -- 172.21.15.106:0/2565202543 >> 172.21.15.106:0/2565202543 conn(0x7f62a809e230 msgr2=0x7f62a80d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 -- 172.21.15.106:0/2565202543 shutdown_connections 2024-04-26T21:46:46.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.443+0000 7f62aee00640 1 -- 172.21.15.106:0/2565202543 wait complete. 2024-04-26T21:46:46.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.445+0000 7f62aee00640 1 Processor -- start 2024-04-26T21:46:46.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.445+0000 7f62aee00640 1 -- start start 2024-04-26T21:46:46.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62900039b0 0x7f62a8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 0x7f62a80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8163210 con 0x7f62a80c8780 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80d33a0 con 0x7f62900039b0 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62aee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62a8163390 con 0x7f62a80ca6f0 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62ad000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62ad000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58018/0 (socket says 172.21.15.106:58018) 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.446+0000 7f62ad000640 1 -- 172.21.15.106:0/3575460941 learned_addr learned my addr 172.21.15.106:0/3575460941 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ada00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62900039b0 0x7f62a8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 -- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 msgr2=0x7f62a80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ae400640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 0x7f62a80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 0x7f62a80caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 -- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62900039b0 msgr2=0x7f62a8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62900039b0 0x7f62a8159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 -- 172.21.15.106:0/3575460941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a80c8c00 con 0x7f62a80c8780 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ae400640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 0x7f62a80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:46.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62ad000640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f629400d800 tx=0x7f629400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294018020 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629400b6c0 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.447+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629400ba90 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.448+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a80c8e80 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.448+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a80c9440 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.449+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a80693c0 con 0x7f62a80c8780 2024-04-26T21:46:46.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.450+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f629401b020 con 0x7f62a80c8780 2024-04-26T21:46:46.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.450+0000 7f62a5600640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 0x7f628806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.451+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f629408bdd0 con 0x7f62a80c8780 2024-04-26T21:46:46.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.451+0000 7f62ada00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 0x7f628806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.451+0000 7f62ada00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 0x7f628806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f629c032000 tx=0x7f629c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.455+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6294022030 con 0x7f62a80c8780 2024-04-26T21:46:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.665+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62a808d090 con 0x7f62a80c8780 2024-04-26T21:46:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.666+0000 7f62a5600640 1 -- 172.21.15.106:0/3575460941 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f62a80c8c00 con 0x7f62a80c8780 2024-04-26T21:46:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:46.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 msgr2=0x7f628806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 0x7f628806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f629c032000 tx=0x7f629c009030 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 msgr2=0x7f62a80ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f629400d800 tx=0x7f629400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 shutdown_connections 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f628806c920 0x7f628806ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62a80ca6f0 0x7f62a80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62a80c8780 0x7f62a80ce690 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 --2- 172.21.15.106:0/3575460941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62900039b0 0x7f62a8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.668+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 >> 172.21.15.106:0/3575460941 conn(0x7f62a809e230 msgr2=0x7f62a80b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.669+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 shutdown_connections 2024-04-26T21:46:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.669+0000 7f62aee00640 1 -- 172.21.15.106:0/3575460941 wait complete. 2024-04-26T21:46:46.685 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:46.686 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:46.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:46.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.927+0000 7fe7d3000640 1 Processor -- start 2024-04-26T21:46:46.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.927+0000 7fe7d3000640 1 -- start start 2024-04-26T21:46:46.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.929+0000 7fe7d3000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe7cc15cb70 0x7fe7cc06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.929+0000 7fe7d3000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7cc15cfd0 0x7fe7cc06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.929+0000 7fe7d3000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7cc0ae850 con 0x7fe7cc06c770 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.929+0000 7fe7d3000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc0af460 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.929+0000 7fe7d3000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc0aeb30 con 0x7fe7cc15cb70 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7cbe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe7cc06c770 0x7fe7cc06cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48620/0 (socket says 172.21.15.106:48620) 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7cbe00640 1 -- 172.21.15.106:0/1147167742 learned_addr learned my addr 172.21.15.106:0/1147167742 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1147167742 >> v2:172.21.15.106:3300/0 conn(0x7fe7cc15cfd0 0x7fe7cc06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7c3e00640 1 --2- 172.21.15.106:0/1147167742 >> v2:172.21.15.204:3300/0 conn(0x7fe7cc15cb70 0x7fe7cc06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7cb400640 1 -- 172.21.15.106:0/1147167742 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593875408 0 0) 0x7fe7cc0ae850 con 0x7fe7cc06c770 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7cb400640 1 -- 172.21.15.106:0/1147167742 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7b0003520 con 0x7fe7cc06c770 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7d1000640 1 -- 172.21.15.106:0/1147167742 >> v2:172.21.15.204:3300/0 conn(0x7fe7cc15cb70 msgr2=0x7fe7cc06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1147167742 >> v2:172.21.15.204:3300/0 conn(0x7fe7cc15cb70 0x7fe7cc06ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7d1000640 1 -- 172.21.15.106:0/1147167742 >> v1:172.21.15.106:6789/0 conn(0x7fe7cc06c770 legacy=0x7fe7cc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.930+0000 7fe7d1000640 1 -- 172.21.15.106:0/1147167742 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc0aee10 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.931+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1147167742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc06f3a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0afec0 tx=0x7fe7c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54af14af1232e50c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7cb400640 1 -- 172.21.15.106:0/1147167742 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4007740 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7cb400640 1 -- 172.21.15.106:0/1147167742 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c40078e0 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7cb400640 1 -- 172.21.15.106:0/1147167742 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4007c90 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 -- 172.21.15.106:0/1147167742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 msgr2=0x7fe7cc06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1147167742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc06f3a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0afec0 tx=0x7fe7c402a680 comp rx=0 tx=0).stop 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 -- 172.21.15.106:0/1147167742 shutdown_connections 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1147167742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc06f3a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1147167742 >> v2:172.21.15.204:3300/0 conn(0x7fe7cc15cb70 0x7fe7cc06ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.932+0000 7fe7d3000640 1 -- 172.21.15.106:0/1147167742 >> 172.21.15.106:0/1147167742 conn(0x7fe7cc0b2d50 msgr2=0x7fe7cc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:46.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.933+0000 7fe7d3000640 1 -- 172.21.15.106:0/1147167742 shutdown_connections 2024-04-26T21:46:46.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.933+0000 7fe7d3000640 1 -- 172.21.15.106:0/1147167742 wait complete. 2024-04-26T21:46:46.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.934+0000 7fe7d3000640 1 Processor -- start 2024-04-26T21:46:46.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.935+0000 7fe7d3000640 1 -- start start 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.935+0000 7fe7d3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 0x7fe7cc14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.935+0000 7fe7d3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 0x7fe7cc0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d3000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d3000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc15c780 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc15c600 con 0x7fe7cc06c770 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc159b90 con 0x7fe7cc15cb70 2024-04-26T21:46:46.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58040/0 (socket says 172.21.15.106:58040) 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 -- 172.21.15.106:0/1609765621 learned_addr learned my addr 172.21.15.106:0/1609765621 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7c3e00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 0x7fe7cc0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7cbe00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 0x7fe7cc14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 -- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 msgr2=0x7fe7cc0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 0x7fe7cc0b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 -- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 msgr2=0x7fe7cc14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 0x7fe7cc14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7d1000640 1 -- 172.21.15.106:0/1609765621 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc1604d0 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7c3e00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 0x7fe7cc0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.936+0000 7fe7cbe00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 0x7fe7cc14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:46.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7d1000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402ac70 tx=0x7fe7c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4040070 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4031bc0 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4009450 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc1601f0 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.937+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc150f40 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.938+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc150a50 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.939+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7c4039040 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.940+0000 7fe7c8c00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 0x7fe7a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:46.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.940+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe7c40ad490 con 0x7fe7cc15cfd0 2024-04-26T21:46:46.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.940+0000 7fe7cbe00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 0x7fe7a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:46.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.941+0000 7fe7cbe00640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 0x7fe7a806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005880 tx=0x7fe7bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:46.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:46.945+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7c407b300 con 0x7fe7cc15cfd0 2024-04-26T21:46:47.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.145+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7cc1593b0 con 0x7fe7cc15cfd0 2024-04-26T21:46:47.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.147+0000 7fe7c8c00640 1 -- 172.21.15.106:0/1609765621 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7fe7c407f600 con 0x7fe7cc15cfd0 2024-04-26T21:46:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:47.148 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:47.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 msgr2=0x7fe7a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 0x7fe7a806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005880 tx=0x7fe7bc009040 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 msgr2=0x7fe7cc15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402ac70 tx=0x7fe7c4004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 shutdown_connections 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7cc15cfd0 0x7fe7cc15f730 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe7a806c8f0 0x7fe7a806edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7cc15cb70 0x7fe7cc0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 --2- 172.21.15.106:0/1609765621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc06c770 0x7fe7cc14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 >> 172.21.15.106:0/1609765621 conn(0x7fe7cc0b2d50 msgr2=0x7fe7cc0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 shutdown_connections 2024-04-26T21:46:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.150+0000 7fe7d3000640 1 -- 172.21.15.106:0/1609765621 wait complete. 2024-04-26T21:46:47.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:47.169 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:47.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.364+0000 7fa531000640 1 Processor -- start 2024-04-26T21:46:47.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.364+0000 7fa531000640 1 -- start start 2024-04-26T21:46:47.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa531000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa52c161140 0x7fa52c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa531000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa52c146280 0x7fa52c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa531000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0b4d20 con 0x7fa52c160d60 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa531000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c146c30 con 0x7fa52c161140 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa531000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0b5000 con 0x7fa52c146280 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa52c161140 0x7fa52c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa52c161140 0x7fa52c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58044/0 (socket says 172.21.15.106:58044) 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 -- 172.21.15.106:0/3871945861 learned_addr learned my addr 172.21.15.106:0/3871945861 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52be00640 1 --2- 172.21.15.106:0/3871945861 >> v2:172.21.15.204:3300/0 conn(0x7fa52c146280 0x7fa52c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 -- 172.21.15.106:0/3871945861 >> v2:172.21.15.204:3300/0 conn(0x7fa52c146280 msgr2=0x7fa52c1466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 --2- 172.21.15.106:0/3871945861 >> v2:172.21.15.204:3300/0 conn(0x7fa52c146280 0x7fa52c1466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.366+0000 7fa52aa00640 1 -- 172.21.15.106:0/3871945861 >> v1:172.21.15.204:6789/0 conn(0x7fa52c160d60 legacy=0x7fa52c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:47.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.367+0000 7fa52aa00640 1 -- 172.21.15.106:0/3871945861 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0b52e0 con 0x7fa52c161140 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa52aa00640 1 --2- 172.21.15.106:0/3871945861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c161140 0x7fa52c06dee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa518009870 tx=0x7fa51802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeed85ce1c26ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa52a000640 1 -- 172.21.15.106:0/3871945861 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518037070 con 0x7fa52c161140 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa52a000640 1 -- 172.21.15.106:0/3871945861 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa518032440 con 0x7fa52c161140 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa52a000640 1 -- 172.21.15.106:0/3871945861 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518032810 con 0x7fa52c161140 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa531000640 1 -- 172.21.15.106:0/3871945861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c161140 msgr2=0x7fa52c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.368+0000 7fa531000640 1 --2- 172.21.15.106:0/3871945861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c161140 0x7fa52c06dee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa518009870 tx=0x7fa51802a270 comp rx=0 tx=0).stop 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 -- 172.21.15.106:0/3871945861 shutdown_connections 2024-04-26T21:46:47.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 --2- 172.21.15.106:0/3871945861 >> v2:172.21.15.204:3300/0 conn(0x7fa52c146280 0x7fa52c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 --2- 172.21.15.106:0/3871945861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c161140 0x7fa52c06dee0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 -- 172.21.15.106:0/3871945861 >> 172.21.15.106:0/3871945861 conn(0x7fa52c0b2900 msgr2=0x7fa52c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:47.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 -- 172.21.15.106:0/3871945861 shutdown_connections 2024-04-26T21:46:47.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.369+0000 7fa531000640 1 -- 172.21.15.106:0/3871945861 wait complete. 2024-04-26T21:46:47.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.371+0000 7fa531000640 1 Processor -- start 2024-04-26T21:46:47.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.371+0000 7fa531000640 1 -- start start 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c146280 0x7fa52c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c161140 0x7fa52c13e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c06e420 con 0x7fa52c160d60 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0cf6f0 con 0x7fa52c146280 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa531000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c06e5a0 con 0x7fa52c161140 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa52aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa52aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58050/0 (socket says 172.21.15.106:58050) 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa52aa00640 1 -- 172.21.15.106:0/3244787234 learned_addr learned my addr 172.21.15.106:0/3244787234 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa52b400640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c146280 0x7fa52c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.372+0000 7fa52be00640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c161140 0x7fa52c13e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 -- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c161140 msgr2=0x7fa52c13e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c161140 0x7fa52c13e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 -- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c146280 msgr2=0x7fa52c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c146280 0x7fa52c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 -- 172.21.15.106:0/3244787234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c158610 con 0x7fa52c160d60 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa52aa00640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa518009870 tx=0x7fa5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518037040 con 0x7fa52c160d60 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa518004230 con 0x7fa52c160d60 2024-04-26T21:46:47.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518003610 con 0x7fa52c160d60 2024-04-26T21:46:47.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.373+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c1588f0 con 0x7fa52c160d60 2024-04-26T21:46:47.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.374+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c15bec0 con 0x7fa52c160d60 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.374+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0693c0 con 0x7fa52c160d60 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.376+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5180037b0 con 0x7fa52c160d60 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.376+0000 7fa523400640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 0x7fa50c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.376+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa5180ac7b0 con 0x7fa52c160d60 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.377+0000 7fa52b400640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 0x7fa50c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.377+0000 7fa52b400640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 0x7fa50c06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa51c004f60 tx=0x7fa51c007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.381+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa51807a500 con 0x7fa52c160d60 2024-04-26T21:46:47.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.566+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa52c15e2f0 con 0x7fa52c160d60 2024-04-26T21:46:47.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.567+0000 7fa523400640 1 -- 172.21.15.106:0/3244787234 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fa51807e800 con 0x7fa52c160d60 2024-04-26T21:46:47.567 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 msgr2=0x7fa50c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 0x7fa50c06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa51c004f60 tx=0x7fa51c007e30 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 msgr2=0x7fa52c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa518009870 tx=0x7fa5180046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 shutdown_connections 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06c920 0x7fa50c06ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c161140 0x7fa52c13e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c160d60 0x7fa52c157ce0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 --2- 172.21.15.106:0/3244787234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c146280 0x7fa52c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.570+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 >> 172.21.15.106:0/3244787234 conn(0x7fa52c0b2900 msgr2=0x7fa52c1442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:47.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.571+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 shutdown_connections 2024-04-26T21:46:47.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.571+0000 7fa531000640 1 -- 172.21.15.106:0/3244787234 wait complete. 2024-04-26T21:46:47.587 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.776+0000 7f6562400640 1 Processor -- start 2024-04-26T21:46:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.776+0000 7f6562400640 1 -- start start 2024-04-26T21:46:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f6562400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 0x7f655c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f6562400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f655c06dfe0 0x7f655c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f6562400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c15d970 con 0x7f655c15cd70 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f6562400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f655c0b4fe0 con 0x7f655c06dfe0 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f6562400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0b52c0 con 0x7f655c15c450 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f655be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f655c06dfe0 0x7f655c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f655be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f655c06dfe0 0x7f655c15c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58054/0 (socket says 172.21.15.106:58054) 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.778+0000 7f655be00640 1 -- 172.21.15.106:0/1687026251 learned_addr learned my addr 172.21.15.106:0/1687026251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f6561000640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 0x7f655c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f655be00640 1 -- 172.21.15.106:0/1687026251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 msgr2=0x7f655c0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f655be00640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 0x7f655c0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f655be00640 1 -- 172.21.15.106:0/1687026251 >> v1:172.21.15.204:6789/0 conn(0x7f655c15cd70 legacy=0x7f655c15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f655be00640 1 -- 172.21.15.106:0/1687026251 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0b55a0 con 0x7f655c06dfe0 2024-04-26T21:46:47.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.779+0000 7f6561000640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 0x7f655c0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.780+0000 7f655be00640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c06dfe0 0x7f655c15c830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f655c071b30 tx=0x7f654402c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6341d79d628bb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.780+0000 7f655b400640 1 -- 172.21.15.106:0/1687026251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544039070 con 0x7f655c06dfe0 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.780+0000 7f655b400640 1 -- 172.21.15.106:0/1687026251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6544034640 con 0x7f655c06dfe0 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.780+0000 7f655b400640 1 -- 172.21.15.106:0/1687026251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544034a10 con 0x7f655c06dfe0 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 -- 172.21.15.106:0/1687026251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c06dfe0 msgr2=0x7f655c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c06dfe0 0x7f655c15c830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f655c071b30 tx=0x7f654402c370 comp rx=0 tx=0).stop 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 -- 172.21.15.106:0/1687026251 shutdown_connections 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c06dfe0 0x7f655c15c830 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 --2- 172.21.15.106:0/1687026251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c15c450 0x7f655c0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 -- 172.21.15.106:0/1687026251 >> 172.21.15.106:0/1687026251 conn(0x7f655c0b2900 msgr2=0x7f655c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 -- 172.21.15.106:0/1687026251 shutdown_connections 2024-04-26T21:46:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.781+0000 7f6562400640 1 -- 172.21.15.106:0/1687026251 wait complete. 2024-04-26T21:46:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.783+0000 7f6562400640 1 Processor -- start 2024-04-26T21:46:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.783+0000 7f6562400640 1 -- start start 2024-04-26T21:46:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c06dfe0 0x7f655c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c15cd70 0x7f655c1f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0b5880 con 0x7f655c15c450 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f655c15d1c0 con 0x7f655c15cd70 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6562400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0b5a00 con 0x7f655c06dfe0 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58066/0 (socket says 172.21.15.106:58066) 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 -- 172.21.15.106:0/344257489 learned_addr learned my addr 172.21.15.106:0/344257489 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6561000640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c06dfe0 0x7f655c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f6561a00640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c15cd70 0x7f655c1f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 -- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c06dfe0 msgr2=0x7f655c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c06dfe0 0x7f655c151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 -- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c15cd70 msgr2=0x7f655c1f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c15cd70 0x7f655c1f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.784+0000 7f655be00640 1 -- 172.21.15.106:0/344257489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0c9f80 con 0x7f655c15c450 2024-04-26T21:46:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f655be00640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544039070 con 0x7f655c15c450 2024-04-26T21:46:47.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6544032980 con 0x7f655c15c450 2024-04-26T21:46:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544032d50 con 0x7f655c15c450 2024-04-26T21:46:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c1575b0 con 0x7f655c15c450 2024-04-26T21:46:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.785+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c157b70 con 0x7f655c15c450 2024-04-26T21:46:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.786+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c0693c0 con 0x7f655c15c450 2024-04-26T21:46:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.787+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f654403f440 con 0x7f655c15c450 2024-04-26T21:46:47.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.788+0000 7f6558c00640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:47.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.788+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f65440ac840 con 0x7f655c15c450 2024-04-26T21:46:47.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.788+0000 7f6561000640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:47.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.789+0000 7f6561000640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 0x7f653806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f654c0075b0 tx=0x7f654c007540 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:47.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.792+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f654407ad00 con 0x7f655c15c450 2024-04-26T21:46:47.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.975+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f655c0d1480 con 0x7f655c15c450 2024-04-26T21:46:47.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.976+0000 7f6558c00640 1 -- 172.21.15.106:0/344257489 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f654407a600 con 0x7f655c15c450 2024-04-26T21:46:47.976 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.978+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 msgr2=0x7f653806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.978+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 0x7f653806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f654c0075b0 tx=0x7f654c007540 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.978+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 msgr2=0x7f655c150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.978+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 shutdown_connections 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c15cd70 0x7f655c1f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c15c450 0x7f655c150f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 --2- 172.21.15.106:0/344257489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c06dfe0 0x7f655c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 >> 172.21.15.106:0/344257489 conn(0x7f655c0b2900 msgr2=0x7f655c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 shutdown_connections 2024-04-26T21:46:47.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:47.979+0000 7f6562400640 1 -- 172.21.15.106:0/344257489 wait complete. 2024-04-26T21:46:47.997 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:48.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.188+0000 7fe20f200640 1 Processor -- start 2024-04-26T21:46:48.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.188+0000 7fe20f200640 1 -- start start 2024-04-26T21:46:48.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe2080b6170 0x7fe2080c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20f200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2080d3830 con 0x7fe2080d1da0 2024-04-26T21:46:48.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe2080d40d0 con 0x7fe2080b6170 2024-04-26T21:46:48.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe2080d3b10 con 0x7fe2080d19c0 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20d200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe2080d1da0 0x7fe208162640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48108/0 (socket says 172.21.15.106:48108) 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.190+0000 7fe20d200640 1 -- 172.21.15.106:0/4010591531 learned_addr learned my addr 172.21.15.106:0/4010591531 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207e00640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207400640 1 --2- 172.21.15.106:0/4010591531 >> v2:172.21.15.204:3300/0 conn(0x7fe2080b6170 0x7fe2080c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378580492 0 0) 0x7fe2080d3830 con 0x7fe2080d1da0 2024-04-26T21:46:48.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ec003520 con 0x7fe2080d1da0 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207400640 1 -- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 msgr2=0x7fe2080d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207400640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207400640 1 -- 172.21.15.106:0/4010591531 >> v1:172.21.15.106:6789/0 conn(0x7fe2080d1da0 legacy=0x7fe208162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1727277260 0 0) 0x7fe1ec003520 con 0x7fe2080d1da0 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.191+0000 7fe207400640 1 -- 172.21.15.106:0/4010591531 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2080d3df0 con 0x7fe2080b6170 2024-04-26T21:46:48.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.193+0000 7fe207400640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080b6170 0x7fe2080c8c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097a0 tx=0x7fe1f002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95a84f4e6369923 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.193+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0037070 con 0x7fe2080b6170 2024-04-26T21:46:48.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.193+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f002ab10 con 0x7fe2080b6170 2024-04-26T21:46:48.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.193+0000 7fe206a00640 1 -- 172.21.15.106:0/4010591531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200001670 con 0x7fe2080b6170 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 -- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080b6170 msgr2=0x7fe2080c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080b6170 0x7fe2080c8c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097a0 tx=0x7fe1f002a080 comp rx=0 tx=0).stop 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 -- 172.21.15.106:0/4010591531 shutdown_connections 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080b6170 0x7fe2080c8c70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 --2- 172.21.15.106:0/4010591531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 -- 172.21.15.106:0/4010591531 >> 172.21.15.106:0/4010591531 conn(0x7fe20809e460 msgr2=0x7fe2080d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 -- 172.21.15.106:0/4010591531 shutdown_connections 2024-04-26T21:46:48.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.194+0000 7fe20f200640 1 -- 172.21.15.106:0/4010591531 wait complete. 2024-04-26T21:46:48.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.196+0000 7fe20f200640 1 Processor -- start 2024-04-26T21:46:48.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.196+0000 7fe20f200640 1 -- start start 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080d1da0 0x7fe2080ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2080cfc30 con 0x7fe2080b6170 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2080c91b0 con 0x7fe2080d1da0 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe20f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe208162d50 con 0x7fe2080d19c0 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53192/0 (socket says 172.21.15.106:53192) 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 -- 172.21.15.106:0/3665992164 learned_addr learned my addr 172.21.15.106:0/3665992164 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 -- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 msgr2=0x7fe2080ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 -- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080d1da0 msgr2=0x7fe2080ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080d1da0 0x7fe2080ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.197+0000 7fe207e00640 1 -- 172.21.15.106:0/3665992164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2080b96c0 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe207e00640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800efc0 tx=0x7fe1f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8018020 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f8014480 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8014850 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2081f5220 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.198+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2080b5a20 con 0x7fe2080b6170 2024-04-26T21:46:48.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.199+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe208069650 con 0x7fe2080b6170 2024-04-26T21:46:48.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.200+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1f8005d10 con 0x7fe2080b6170 2024-04-26T21:46:48.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.201+0000 7fe1ffe00640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 0x7fe1e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.201+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe1f808c7a0 con 0x7fe2080b6170 2024-04-26T21:46:48.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.201+0000 7fe207400640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 0x7fe1e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.202+0000 7fe207400640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 0x7fe1e406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0034000 tx=0x7fe1f002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.209+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1f8022070 con 0x7fe2080b6170 2024-04-26T21:46:48.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.408+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2080d26d0 con 0x7fe2080b6170 2024-04-26T21:46:48.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.409+0000 7fe1ffe00640 1 -- 172.21.15.106:0/3665992164 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fe1f8010040 con 0x7fe2080b6170 2024-04-26T21:46:48.409 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.411+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 msgr2=0x7fe1e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.411+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 0x7fe1e406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0034000 tx=0x7fe1f002c000 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.411+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 msgr2=0x7fe208155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.411+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800efc0 tx=0x7fe1f800c490 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 shutdown_connections 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2080d1da0 0x7fe2080ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe1e406c8f0 0x7fe1e406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2080d19c0 0x7fe2080ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 --2- 172.21.15.106:0/3665992164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2080b6170 0x7fe208155370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 >> 172.21.15.106:0/3665992164 conn(0x7fe20809e460 msgr2=0x7fe2080cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 shutdown_connections 2024-04-26T21:46:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.412+0000 7fe20f200640 1 -- 172.21.15.106:0/3665992164 wait complete. 2024-04-26T21:46:48.429 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:48.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.613+0000 7fc398400640 1 Processor -- start 2024-04-26T21:46:48.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.614+0000 7fc398400640 1 -- start start 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc398400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3901598a0 0x7fc3901638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc398400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc39006c770 0x7fc39006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc398400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3900b5fb0 con 0x7fc39015f740 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc398400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc390159c80 con 0x7fc39006c770 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc398400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc39006d110 con 0x7fc3901598a0 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc395000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc39006c770 0x7fc39006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc396400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc39015f740 0x7fc3900add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48110/0 (socket says 172.21.15.106:48110) 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.615+0000 7fc396400640 1 -- 172.21.15.106:0/1889232875 learned_addr learned my addr 172.21.15.106:0/1889232875 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc395a00640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3901598a0 0x7fc3901638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc395000640 1 -- 172.21.15.106:0/1889232875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3901598a0 msgr2=0x7fc3901638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc395000640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3901598a0 0x7fc3901638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc395000640 1 -- 172.21.15.106:0/1889232875 >> v1:172.21.15.106:6789/0 conn(0x7fc39015f740 legacy=0x7fc3900add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc38fe00640 1 -- 172.21.15.106:0/1889232875 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609752368 0 0) 0x7fc3900b5fb0 con 0x7fc39015f740 2024-04-26T21:46:48.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.616+0000 7fc395000640 1 -- 172.21.15.106:0/1889232875 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc390160370 con 0x7fc39006c770 2024-04-26T21:46:48.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.617+0000 7fc395000640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc39006c770 0x7fc39006cbd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0097a0 tx=0x7fc37c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99155709abb7af76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.617+0000 7fc38fe00640 1 -- 172.21.15.106:0/1889232875 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c037070 con 0x7fc39006c770 2024-04-26T21:46:48.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.617+0000 7fc38fe00640 1 -- 172.21.15.106:0/1889232875 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c02ae50 con 0x7fc39006c770 2024-04-26T21:46:48.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.617+0000 7fc38fe00640 1 -- 172.21.15.106:0/1889232875 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c032660 con 0x7fc39006c770 2024-04-26T21:46:48.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 -- 172.21.15.106:0/1889232875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc39006c770 msgr2=0x7fc39006cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc39006c770 0x7fc39006cbd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0097a0 tx=0x7fc37c02a100 comp rx=0 tx=0).stop 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 -- 172.21.15.106:0/1889232875 shutdown_connections 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc39006c770 0x7fc39006cbd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 --2- 172.21.15.106:0/1889232875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3901598a0 0x7fc3901638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 -- 172.21.15.106:0/1889232875 >> 172.21.15.106:0/1889232875 conn(0x7fc3900b2d50 msgr2=0x7fc3900b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 -- 172.21.15.106:0/1889232875 shutdown_connections 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.618+0000 7fc398400640 1 -- 172.21.15.106:0/1889232875 wait complete. 2024-04-26T21:46:48.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.620+0000 7fc398400640 1 Processor -- start 2024-04-26T21:46:48.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.620+0000 7fc398400640 1 -- start start 2024-04-26T21:46:48.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39006c770 0x7fc390141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc39015f740 0x7fc39009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc390163390 con 0x7fc3901598a0 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901630b0 con 0x7fc39006c770 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc398400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc390160650 con 0x7fc39015f740 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53202/0 (socket says 172.21.15.106:53202) 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 -- 172.21.15.106:0/1800656444 learned_addr learned my addr 172.21.15.106:0/1800656444 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 -- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc39015f740 msgr2=0x7fc39009cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc39015f740 0x7fc39009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 -- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39006c770 msgr2=0x7fc390141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39006c770 0x7fc390141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.621+0000 7fc395000640 1 -- 172.21.15.106:0/1800656444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900ca270 con 0x7fc3901598a0 2024-04-26T21:46:48.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc395000640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098c0 tx=0x7fc37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c042070 con 0x7fc3901598a0 2024-04-26T21:46:48.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c03e670 con 0x7fc3901598a0 2024-04-26T21:46:48.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c037040 con 0x7fc3901598a0 2024-04-26T21:46:48.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3900ca550 con 0x7fc3901598a0 2024-04-26T21:46:48.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.622+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3900cab10 con 0x7fc3901598a0 2024-04-26T21:46:48.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.623+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390069540 con 0x7fc3901598a0 2024-04-26T21:46:48.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.624+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc37c045070 con 0x7fc3901598a0 2024-04-26T21:46:48.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.625+0000 7fc38d600640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:48.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.625+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc37c0ac3c0 con 0x7fc3901598a0 2024-04-26T21:46:48.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.625+0000 7fc395a00640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:48.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.626+0000 7fc395a00640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 0x7fc37006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:48.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.630+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc37c07a760 con 0x7fc3901598a0 2024-04-26T21:46:48.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.841+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc390162330 con 0x7fc3901598a0 2024-04-26T21:46:48.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.842+0000 7fc38d600640 1 -- 172.21.15.106:0/1800656444 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc37c07a060 con 0x7fc3901598a0 2024-04-26T21:46:48.842 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:48.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 msgr2=0x7fc37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 0x7fc37006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384008040 comp rx=0 tx=0).stop 2024-04-26T21:46:48.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 msgr2=0x7fc3900cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098c0 tx=0x7fc37c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 shutdown_connections 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc39015f740 0x7fc39009cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3901598a0 0x7fc3900cf5c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 --2- 172.21.15.106:0/1800656444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39006c770 0x7fc390141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 >> 172.21.15.106:0/1800656444 conn(0x7fc3900b2d50 msgr2=0x7fc3900b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 shutdown_connections 2024-04-26T21:46:48.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:48.845+0000 7fc398400640 1 -- 172.21.15.106:0/1800656444 wait complete. 2024-04-26T21:46:48.865 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-04-26T21:46:48.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:48.865 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T21:46:48.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:46:48.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:46:48.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:48.962 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:46:49.023 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:49.023 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:49.081 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:49.081 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:49.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:49.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.328+0000 7f1155000640 1 Processor -- start 2024-04-26T21:46:49.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.328+0000 7f1155000640 1 -- start start 2024-04-26T21:46:49.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.329+0000 7f1155000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11500add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.329+0000 7f1155000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11500af9a0 con 0x7f11500cd650 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.329+0000 7f1155000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11500afc80 con 0x7f115006ce80 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.329+0000 7f1155000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f11500ae2c0 con 0x7f115015cdf0 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.330+0000 7f114d400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f11500cd650 0x7f115006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48120/0 (socket says 172.21.15.106:48120) 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.330+0000 7f114d400640 1 -- 172.21.15.106:0/2593616996 learned_addr learned my addr 172.21.15.106:0/2593616996 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.330+0000 7f114de00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11500add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.330+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247867728 0 0) 0x7f11500af9a0 con 0x7f11500cd650 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.330+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1134003520 con 0x7f11500cd650 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4228264428 0 0) 0x7f1134003520 con 0x7f11500cd650 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 msgr2=0x7f11500add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11500add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 >> v1:172.21.15.204:6789/0 conn(0x7f115006ce80 legacy=0x7f11500b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11500aff60 con 0x7f11500cd650 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f113c001670 con 0x7f11500cd650 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 >> v1:172.21.15.106:6789/0 conn(0x7f11500cd650 legacy=0x7f115006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f1134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f113400ba70 con 0x7f1134003f60 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f113400bc20 con 0x7f1134004d40 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f113400be20 con 0x7f115015cdf0 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.331+0000 7f114d400640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f1134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114e800640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114de00640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114d400640 1 -- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 msgr2=0x7f11340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114d400640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115015cdf0 0x7f11340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114d400640 1 -- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1134004d40 msgr2=0x7f11340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114d400640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.332+0000 7f114d400640 1 -- 172.21.15.106:0/2593616996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134003520 con 0x7f1134003f60 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.333+0000 7f114d400640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f1134004800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c240 tx=0x7f1134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e7a102c85c7db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.333+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c007cf0 con 0x7f1134003f60 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.333+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c0058d0 con 0x7f1134003f60 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.333+0000 7f114ca00640 1 -- 172.21.15.106:0/2593616996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c005cc0 con 0x7f1134003f60 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 -- 172.21.15.106:0/2593616996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 msgr2=0x7f1134004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 --2- 172.21.15.106:0/2593616996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f1134004800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c240 tx=0x7f1134005830 comp rx=0 tx=0).stop 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f114e800640 1 -- 172.21.15.106:0/2593616996 reap_dead start 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 -- 172.21.15.106:0/2593616996 shutdown_connections 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 -- 172.21.15.106:0/2593616996 >> 172.21.15.106:0/2593616996 conn(0x7f11500b2d50 msgr2=0x7f11500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 -- 172.21.15.106:0/2593616996 shutdown_connections 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.334+0000 7f1155000640 1 -- 172.21.15.106:0/2593616996 wait complete. 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.336+0000 7f1155000640 1 Processor -- start 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.337+0000 7f1155000640 1 -- start start 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 0x7f1150156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11500cb550 0x7f11500cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f115015d1d0 con 0x7f1134003f60 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11500b05b0 con 0x7f11500cb550 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f1155000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f115015d350 con 0x7f1150155ea0 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f114de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f114de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53222/0 (socket says 172.21.15.106:53222) 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.338+0000 7f114de00640 1 -- 172.21.15.106:0/3010522028 learned_addr learned my addr 172.21.15.106:0/3010522028 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114d400640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 0x7f1150156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 -- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 msgr2=0x7f1150156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 0x7f1150156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 -- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11500cb550 msgr2=0x7f11500cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114e800640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11500cb550 0x7f11500cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11500cb550 0x7f11500cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 -- 172.21.15.106:0/3010522028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115015a230 con 0x7f1134003f60 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114d400640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 0x7f1150156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:49.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f114de00640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1140011600 tx=0x7f1140011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114000f8c0 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114000fa60 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114000fe10 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.339+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115015a510 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.340+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115015aad0 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.341+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1150069540 con 0x7f1134003f60 2024-04-26T21:46:49.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.342+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1140015460 con 0x7f1134003f60 2024-04-26T21:46:49.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.343+0000 7f1146000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 0x7f113006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.343+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f114008c610 con 0x7f1134003f60 2024-04-26T21:46:49.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.343+0000 7f114d400640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 0x7f113006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.344+0000 7f114d400640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 0x7f113006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f113c033000 tx=0x7f113c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.347+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1140019030 con 0x7f1134003f60 2024-04-26T21:46:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.547+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11500b1fb0 con 0x7f1134003f60 2024-04-26T21:46:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.550+0000 7f1146000640 1 -- 172.21.15.106:0/3010522028 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f114005a920 con 0x7f1134003f60 2024-04-26T21:46:49.551 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:49.551 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 msgr2=0x7f113006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 0x7f113006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f113c033000 tx=0x7f113c009030 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 msgr2=0x7f115015b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1140011600 tx=0x7f1140011ac0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 shutdown_connections 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11500cb550 0x7f11500cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f113006c8f0 0x7f113006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1150155ea0 0x7f1150156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 --2- 172.21.15.106:0/3010522028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1134003f60 0x7f115015b2d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.553+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 >> 172.21.15.106:0/3010522028 conn(0x7f11500b2d50 msgr2=0x7f11500b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.554+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 shutdown_connections 2024-04-26T21:46:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.554+0000 7f1155000640 1 -- 172.21.15.106:0/3010522028 wait complete. 2024-04-26T21:46:49.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:49.574 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:49.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.778+0000 7f89bea00640 1 Processor -- start 2024-04-26T21:46:49.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.778+0000 7f89bea00640 1 -- start start 2024-04-26T21:46:49.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.779+0000 7f89bea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f89b8160d60 0x7f89b80b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.779+0000 7f89bea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b80b4d20 con 0x7f89b8161140 2024-04-26T21:46:49.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b80b5000 con 0x7f89b8146280 2024-04-26T21:46:49.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bea00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f89b806e5f0 con 0x7f89b8160d60 2024-04-26T21:46:49.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bd600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f89b8160d60 0x7f89b80b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bcc00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f89b8161140 0x7f89b806dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48132/0 (socket says 172.21.15.106:48132) 2024-04-26T21:46:49.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bcc00640 1 -- 172.21.15.106:0/1128545553 learned_addr learned my addr 172.21.15.106:0/1128545553 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:49.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bd600640 1 -- 172.21.15.106:0/1128545553 >> v1:172.21.15.204:6789/0 conn(0x7f89b8146280 legacy=0x7f89b80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bd600640 1 -- 172.21.15.106:0/1128545553 >> v1:172.21.15.106:6789/0 conn(0x7f89b8161140 legacy=0x7f89b806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89bd600640 1 -- 172.21.15.106:0/1128545553 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80b52e0 con 0x7f89b8160d60 2024-04-26T21:46:49.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.780+0000 7f89b7e00640 1 -- 172.21.15.106:0/1128545553 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308296411 0 0) 0x7f89b80b4d20 con 0x7f89b8161140 2024-04-26T21:46:49.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.781+0000 7f89bd600640 1 --2- 172.21.15.106:0/1128545553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8160d60 0x7f89b80b5b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0097d0 tx=0x7f89ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3776667fa073c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89b7e00640 1 -- 172.21.15.106:0/1128545553 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac037070 con 0x7f89b8160d60 2024-04-26T21:46:49.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89b7e00640 1 -- 172.21.15.106:0/1128545553 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac032440 con 0x7f89b8160d60 2024-04-26T21:46:49.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89b7e00640 1 -- 172.21.15.106:0/1128545553 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac032810 con 0x7f89b8160d60 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 -- 172.21.15.106:0/1128545553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8160d60 msgr2=0x7f89b80b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 --2- 172.21.15.106:0/1128545553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8160d60 0x7f89b80b5b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0097d0 tx=0x7f89ac02a210 comp rx=0 tx=0).stop 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 -- 172.21.15.106:0/1128545553 shutdown_connections 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 --2- 172.21.15.106:0/1128545553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8160d60 0x7f89b80b5b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 -- 172.21.15.106:0/1128545553 >> 172.21.15.106:0/1128545553 conn(0x7f89b80b2900 msgr2=0x7f89b80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.782+0000 7f89bea00640 1 -- 172.21.15.106:0/1128545553 shutdown_connections 2024-04-26T21:46:49.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.783+0000 7f89bea00640 1 -- 172.21.15.106:0/1128545553 wait complete. 2024-04-26T21:46:49.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.784+0000 7f89bea00640 1 Processor -- start 2024-04-26T21:46:49.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.784+0000 7f89bea00640 1 -- start start 2024-04-26T21:46:49.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89b8160d60 0x7f89b8136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89b8161140 0x7f89b815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80b6070 con 0x7f89b8146280 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80cf6f0 con 0x7f89b8161140 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.785+0000 7f89bea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f89b8161770 con 0x7f89b8160d60 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53246/0 (socket says 172.21.15.106:53246) 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 -- 172.21.15.106:0/3501007643 learned_addr learned my addr 172.21.15.106:0/3501007643 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 -- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89b8160d60 msgr2=0x7f89b8136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89b8160d60 0x7f89b8136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 -- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89b8161140 msgr2=0x7f89b815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89be000640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89b8161140 0x7f89b815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89b8161140 0x7f89b815ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 -- 172.21.15.106:0/3501007643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b81377f0 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.786+0000 7f89bd600640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0098f0 tx=0x7f89ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.787+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac037040 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.787+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac004230 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.787+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac03c730 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.787+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b8137ad0 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.787+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b8138090 con 0x7f89b8146280 2024-04-26T21:46:49.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.788+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b80693c0 con 0x7f89b8146280 2024-04-26T21:46:49.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.789+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89ac03a070 con 0x7f89b8146280 2024-04-26T21:46:49.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.790+0000 7f89b5600640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.790+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f89ac0ac590 con 0x7f89b8146280 2024-04-26T21:46:49.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.790+0000 7f89bcc00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.791+0000 7f89bcc00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 0x7f899806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89a4007fd0 tx=0x7f89a4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.794+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89ac041020 con 0x7f89b8146280 2024-04-26T21:46:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.992+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89b80d1860 con 0x7f89b8146280 2024-04-26T21:46:49.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.995+0000 7f89b5600640 1 -- 172.21.15.106:0/3501007643 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f89ac07a930 con 0x7f89b8146280 2024-04-26T21:46:49.995 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:49.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 msgr2=0x7f899806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 0x7f899806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89a4007fd0 tx=0x7f89a4007920 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 msgr2=0x7f89b81599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0098f0 tx=0x7f89ac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 shutdown_connections 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89b8161140 0x7f89b815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89b8160d60 0x7f89b8136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 --2- 172.21.15.106:0/3501007643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b8146280 0x7f89b81599c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 >> 172.21.15.106:0/3501007643 conn(0x7f89b80b2900 msgr2=0x7f89b81457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 shutdown_connections 2024-04-26T21:46:49.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:49.998+0000 7f89bea00640 1 -- 172.21.15.106:0/3501007643 wait complete. 2024-04-26T21:46:50.018 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:50.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.219+0000 7f301e600640 1 Processor -- start 2024-04-26T21:46:50.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.219+0000 7f301e600640 1 -- start start 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f301e600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f301815d100 0x7f301806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f301e600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30180add80 0x7f301806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f301e600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30180af800 con 0x7f30180b5ca0 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f301e600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f30180ae340 con 0x7f301815d100 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f301e600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f30180afaa0 con 0x7f30180add80 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f3017400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f301815d100 0x7f301806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f3016a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f30180b5ca0 0x7f3018072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48138/0 (socket says 172.21.15.106:48138) 2024-04-26T21:46:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f3016a00640 1 -- 172.21.15.106:0/1133790606 learned_addr learned my addr 172.21.15.106:0/1133790606 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.221+0000 7f3017e00640 1 --2- 172.21.15.106:0/1133790606 >> v2:172.21.15.204:3300/0 conn(0x7f30180add80 0x7f301806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.222+0000 7f3017400640 1 -- 172.21.15.106:0/1133790606 >> v2:172.21.15.204:3300/0 conn(0x7f30180add80 msgr2=0x7f301806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.222+0000 7f3017400640 1 --2- 172.21.15.106:0/1133790606 >> v2:172.21.15.204:3300/0 conn(0x7f30180add80 0x7f301806ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.222+0000 7f3017400640 1 -- 172.21.15.106:0/1133790606 >> v1:172.21.15.106:6789/0 conn(0x7f30180b5ca0 legacy=0x7f3018072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.222+0000 7f3017400640 1 -- 172.21.15.106:0/1133790606 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180afd80 con 0x7f301815d100 2024-04-26T21:46:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.222+0000 7f3016000640 1 -- 172.21.15.106:0/1133790606 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582272205 0 0) 0x7f30180af800 con 0x7f30180b5ca0 2024-04-26T21:46:50.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.223+0000 7f3017400640 1 --2- 172.21.15.106:0/1133790606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301815d100 0x7f301806c770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3018073560 tx=0x7f300802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc09d5e3e52355 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.223+0000 7f3016000640 1 -- 172.21.15.106:0/1133790606 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037070 con 0x7f301815d100 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.223+0000 7f3016000640 1 -- 172.21.15.106:0/1133790606 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3008032440 con 0x7f301815d100 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.223+0000 7f3016000640 1 -- 172.21.15.106:0/1133790606 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008032810 con 0x7f301815d100 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 -- 172.21.15.106:0/1133790606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301815d100 msgr2=0x7f301806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 --2- 172.21.15.106:0/1133790606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301815d100 0x7f301806c770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3018073560 tx=0x7f300802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 -- 172.21.15.106:0/1133790606 shutdown_connections 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 --2- 172.21.15.106:0/1133790606 >> v2:172.21.15.204:3300/0 conn(0x7f30180add80 0x7f301806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 --2- 172.21.15.106:0/1133790606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301815d100 0x7f301806c770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 -- 172.21.15.106:0/1133790606 >> 172.21.15.106:0/1133790606 conn(0x7f30180b2d50 msgr2=0x7f30180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 -- 172.21.15.106:0/1133790606 shutdown_connections 2024-04-26T21:46:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.224+0000 7f301e600640 1 -- 172.21.15.106:0/1133790606 wait complete. 2024-04-26T21:46:50.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.226+0000 7f301e600640 1 Processor -- start 2024-04-26T21:46:50.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.226+0000 7f301e600640 1 -- start start 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30180b5ca0 0x7f301815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f301815d100 0x7f301815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f301809ee50 con 0x7f30180add80 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30180b0340 con 0x7f301815d100 2024-04-26T21:46:50.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f301e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3018160b70 con 0x7f30180b5ca0 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53264/0 (socket says 172.21.15.106:53264) 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 -- 172.21.15.106:0/1748593165 learned_addr learned my addr 172.21.15.106:0/1748593165 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 -- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30180b5ca0 msgr2=0x7f301815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30180b5ca0 0x7f301815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 -- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f301815d100 msgr2=0x7f301815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f301815d100 0x7f301815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.227+0000 7f3017400640 1 -- 172.21.15.106:0/1748593165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301814bcc0 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f3017400640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f30080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037040 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3008042070 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008030bd0 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301815e090 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.228+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301815e650 con 0x7f30180add80 2024-04-26T21:46:50.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.229+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3018069650 con 0x7f30180add80 2024-04-26T21:46:50.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.230+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3008047460 con 0x7f30180add80 2024-04-26T21:46:50.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.231+0000 7f300f400640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 0x7f2ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.231+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f30080ac7b0 con 0x7f30180add80 2024-04-26T21:46:50.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.231+0000 7f3016a00640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 0x7f2ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.232+0000 7f3016a00640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 0x7f2ff806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3004007fd0 tx=0x7f3004007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.236+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f301814bcc0 con 0x7f30180add80 2024-04-26T21:46:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.439+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3018073d00 con 0x7f30180add80 2024-04-26T21:46:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.440+0000 7f300f400640 1 -- 172.21.15.106:0/1748593165 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3008032e20 con 0x7f30180add80 2024-04-26T21:46:50.440 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 msgr2=0x7f2ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 0x7f2ff806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3004007fd0 tx=0x7f3004007920 comp rx=0 tx=0).stop 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 msgr2=0x7f301814b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f30080046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 shutdown_connections 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f301815d100 0x7f301815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2ff806c8f0 0x7f2ff806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30180b5ca0 0x7f301815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 --2- 172.21.15.106:0/1748593165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30180add80 0x7f301814b780 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 >> 172.21.15.106:0/1748593165 conn(0x7f30180b2d50 msgr2=0x7f30180b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 shutdown_connections 2024-04-26T21:46:50.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.443+0000 7f301e600640 1 -- 172.21.15.106:0/1748593165 wait complete. 2024-04-26T21:46:50.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:50.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.665+0000 7fcb37a00640 1 Processor -- start 2024-04-26T21:46:50.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.665+0000 7fcb37a00640 1 -- start start 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb37a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcb30156650 0x7fcb3015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb37a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb3006c770 0x7fcb3006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb37a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3006eea0 con 0x7fcb30073120 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb37a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcb300b4c40 con 0x7fcb3006c770 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb37a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3006f140 con 0x7fcb30156650 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb3006c770 0x7fcb3006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb3006c770 0x7fcb3006cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53268/0 (socket says 172.21.15.106:53268) 2024-04-26T21:46:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 -- 172.21.15.106:0/3901886855 learned_addr learned my addr 172.21.15.106:0/3901886855 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:50.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb35000640 1 --2- 172.21.15.106:0/3901886855 >> v2:172.21.15.204:3300/0 conn(0x7fcb30156650 0x7fcb3015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 -- 172.21.15.106:0/3901886855 >> v2:172.21.15.204:3300/0 conn(0x7fcb30156650 msgr2=0x7fcb3015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 --2- 172.21.15.106:0/3901886855 >> v2:172.21.15.204:3300/0 conn(0x7fcb30156650 0x7fcb3015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.667+0000 7fcb27e00640 1 -- 172.21.15.106:0/3901886855 >> v1:172.21.15.204:6789/0 conn(0x7fcb30073120 legacy=0x7fcb30073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.668+0000 7fcb27e00640 1 -- 172.21.15.106:0/3901886855 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3006f420 con 0x7fcb3006c770 2024-04-26T21:46:50.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb27e00640 1 --2- 172.21.15.106:0/3901886855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3006c770 0x7fcb3006cbf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb30075cd0 tx=0x7fcb1802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0141f9d12012bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb2fe00640 1 -- 172.21.15.106:0/3901886855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb180325a0 con 0x7fcb3006c770 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb2fe00640 1 -- 172.21.15.106:0/3901886855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb18038070 con 0x7fcb3006c770 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb2fe00640 1 -- 172.21.15.106:0/3901886855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18032960 con 0x7fcb3006c770 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb37a00640 1 -- 172.21.15.106:0/3901886855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3006c770 msgr2=0x7fcb3006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.669+0000 7fcb37a00640 1 --2- 172.21.15.106:0/3901886855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3006c770 0x7fcb3006cbf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb30075cd0 tx=0x7fcb1802c8e0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 -- 172.21.15.106:0/3901886855 shutdown_connections 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 --2- 172.21.15.106:0/3901886855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3006c770 0x7fcb3006cbf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 --2- 172.21.15.106:0/3901886855 >> v2:172.21.15.204:3300/0 conn(0x7fcb30156650 0x7fcb3015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 -- 172.21.15.106:0/3901886855 >> 172.21.15.106:0/3901886855 conn(0x7fcb300b2d50 msgr2=0x7fcb300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 -- 172.21.15.106:0/3901886855 shutdown_connections 2024-04-26T21:46:50.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.670+0000 7fcb37a00640 1 -- 172.21.15.106:0/3901886855 wait complete. 2024-04-26T21:46:50.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.671+0000 7fcb37a00640 1 Processor -- start 2024-04-26T21:46:50.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 -- start start 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb3006c770 0x7fcb30146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb30156650 0x7fcb30154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3006fcc0 con 0x7fcb30073120 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3006f9e0 con 0x7fcb30156650 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.672+0000 7fcb37a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3015d320 con 0x7fcb3006c770 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53270/0 (socket says 172.21.15.106:53270) 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 -- 172.21.15.106:0/2568384697 learned_addr learned my addr 172.21.15.106:0/2568384697 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 -- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb3006c770 msgr2=0x7fcb30146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb35000640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb3006c770 0x7fcb30146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb3006c770 0x7fcb30146600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 -- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb30156650 msgr2=0x7fcb30154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb30156650 0x7fcb30154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.673+0000 7fcb27e00640 1 -- 172.21.15.106:0/2568384697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb30159ff0 con 0x7fcb30073120 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb27e00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18043070 con 0x7fcb30073120 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb18033aa0 con 0x7fcb30073120 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1803c420 con 0x7fcb30073120 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3015a2d0 con 0x7fcb30073120 2024-04-26T21:46:50.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.674+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3015a890 con 0x7fcb30073120 2024-04-26T21:46:50.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.675+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30069650 con 0x7fcb30073120 2024-04-26T21:46:50.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.676+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb18046070 con 0x7fcb30073120 2024-04-26T21:46:50.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.678+0000 7fcb2d600640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.678+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcb18030020 con 0x7fcb30073120 2024-04-26T21:46:50.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.678+0000 7fcb35000640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.679+0000 7fcb35000640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.682+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb1804d070 con 0x7fcb30073120 2024-04-26T21:46:50.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.883+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb3006ec80 con 0x7fcb30073120 2024-04-26T21:46:50.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.884+0000 7fcb2d600640 1 -- 172.21.15.106:0/2568384697 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fcb1807a8b0 con 0x7fcb30073120 2024-04-26T21:46:50.884 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:50.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 msgr2=0x7fcb0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20008040 comp rx=0 tx=0).stop 2024-04-26T21:46:50.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 msgr2=0x7fcb300ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb18004730 comp rx=0 tx=0).stop 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 shutdown_connections 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb30156650 0x7fcb30154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb30073120 0x7fcb300ae740 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 --2- 172.21.15.106:0/2568384697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb3006c770 0x7fcb30146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 >> 172.21.15.106:0/2568384697 conn(0x7fcb300b2d50 msgr2=0x7fcb30152aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 shutdown_connections 2024-04-26T21:46:50.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:50.887+0000 7fcb37a00640 1 -- 172.21.15.106:0/2568384697 wait complete. 2024-04-26T21:46:50.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:51.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.107+0000 7f0a7bc00640 1 Processor -- start 2024-04-26T21:46:51.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.108+0000 7f0a7bc00640 1 -- start start 2024-04-26T21:46:51.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a7bc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a7bc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740b5d30 con 0x7f0a7415ef60 2024-04-26T21:46:51.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a7bc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740b6010 con 0x7f0a740d34a0 2024-04-26T21:46:51.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a7bc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0a740b62f0 con 0x7f0a740d5d90 2024-04-26T21:46:51.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a79200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0a7415ef60 0x7f0a740b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48154/0 (socket says 172.21.15.106:48154) 2024-04-26T21:46:51.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a79200640 1 -- 172.21.15.106:0/1417013587 learned_addr learned my addr 172.21.15.106:0/1417013587 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:51.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.109+0000 7f0a6be00640 1 --2- 172.21.15.106:0/1417013587 >> v2:172.21.15.106:3300/0 conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.110+0000 7f0a73e00640 1 -- 172.21.15.106:0/1417013587 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110154849 0 0) 0x7f0a740b5d30 con 0x7f0a7415ef60 2024-04-26T21:46:51.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.110+0000 7f0a73e00640 1 -- 172.21.15.106:0/1417013587 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a58003520 con 0x7f0a7415ef60 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.110+0000 7f0a6be00640 1 -- 172.21.15.106:0/1417013587 >> v1:172.21.15.204:6789/0 conn(0x7f0a740d34a0 legacy=0x7f0a740d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.110+0000 7f0a6be00640 1 -- 172.21.15.106:0/1417013587 >> v1:172.21.15.106:6789/0 conn(0x7f0a7415ef60 legacy=0x7f0a740b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.110+0000 7f0a6be00640 1 -- 172.21.15.106:0/1417013587 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740b65d0 con 0x7f0a740d5d90 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.111+0000 7f0a6be00640 1 --2- 172.21.15.106:0/1417013587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097a0 tx=0x7f0a5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db285a5d7feb68d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.111+0000 7f0a73e00640 1 -- 172.21.15.106:0/1417013587 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c037070 con 0x7f0a740d5d90 2024-04-26T21:46:51.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.111+0000 7f0a73e00640 1 -- 172.21.15.106:0/1417013587 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5c032440 con 0x7f0a740d5d90 2024-04-26T21:46:51.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.111+0000 7f0a73e00640 1 -- 172.21.15.106:0/1417013587 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c032810 con 0x7f0a740d5d90 2024-04-26T21:46:51.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 -- 172.21.15.106:0/1417013587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 msgr2=0x7f0a7415f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/1417013587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097a0 tx=0x7f0a5c02a270 comp rx=0 tx=0).stop 2024-04-26T21:46:51.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 -- 172.21.15.106:0/1417013587 shutdown_connections 2024-04-26T21:46:51.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/1417013587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 -- 172.21.15.106:0/1417013587 >> 172.21.15.106:0/1417013587 conn(0x7f0a7409e3b0 msgr2=0x7f0a740d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 -- 172.21.15.106:0/1417013587 shutdown_connections 2024-04-26T21:46:51.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.112+0000 7f0a7bc00640 1 -- 172.21.15.106:0/1417013587 wait complete. 2024-04-26T21:46:51.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.114+0000 7f0a7bc00640 1 Processor -- start 2024-04-26T21:46:51.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.114+0000 7f0a7bc00640 1 -- start start 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 0x7f0a740ba260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 0x7f0a74161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740d3ab0 con 0x7f0a740d5d90 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740d3930 con 0x7f0a7415ef60 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a740d3c30 con 0x7f0a740d34a0 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53292/0 (socket says 172.21.15.106:53292) 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.115+0000 7f0a6be00640 1 -- 172.21.15.106:0/342366560 learned_addr learned my addr 172.21.15.106:0/342366560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 -- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 msgr2=0x7f0a740ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a79200640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 0x7f0a740ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a79c00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 0x7f0a74161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 0x7f0a740ba260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 -- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 msgr2=0x7f0a74161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 0x7f0a74161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 -- 172.21.15.106:0/342366560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7415f880 con 0x7f0a740d5d90 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a79c00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 0x7f0a74161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a79200640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 0x7f0a740ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:51.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a6be00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000c00 tx=0x7f0a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c037040 con 0x7f0a740d5d90 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5c042070 con 0x7f0a740d5d90 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c030940 con 0x7f0a740d5d90 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.116+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7415ffa0 con 0x7f0a740d5d90 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.117+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a740cd2c0 con 0x7f0a740d5d90 2024-04-26T21:46:51.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.118+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a740cd790 con 0x7f0a740d5d90 2024-04-26T21:46:51.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.118+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a5c030ae0 con 0x7f0a740d5d90 2024-04-26T21:46:51.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.119+0000 7f0a71600640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 0x7f0a5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.119+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0a5c0ac630 con 0x7f0a740d5d90 2024-04-26T21:46:51.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.120+0000 7f0a79200640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 0x7f0a5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.120+0000 7f0a79200640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 0x7f0a5006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a64005c00 tx=0x7f0a64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.124+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a5c045360 con 0x7f0a740d5d90 2024-04-26T21:46:51.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.321+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a74069540 con 0x7f0a740d5d90 2024-04-26T21:46:51.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.322+0000 7f0a71600640 1 -- 172.21.15.106:0/342366560 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f0a5c07a8b0 con 0x7f0a740d5d90 2024-04-26T21:46:51.322 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 msgr2=0x7f0a5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 0x7f0a5006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a64005c00 tx=0x7f0a64009040 comp rx=0 tx=0).stop 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 msgr2=0x7f0a7415f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000c00 tx=0x7f0a5c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 shutdown_connections 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a7415ef60 0x7f0a74161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a740d5d90 0x7f0a7415f340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0a5006c920 0x7f0a5006ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 --2- 172.21.15.106:0/342366560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a740d34a0 0x7f0a740ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 >> 172.21.15.106:0/342366560 conn(0x7f0a7409e3b0 msgr2=0x7f0a7406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 shutdown_connections 2024-04-26T21:46:51.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.324+0000 7f0a7bc00640 1 -- 172.21.15.106:0/342366560 wait complete. 2024-04-26T21:46:51.342 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:46:51.342 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:51.391 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:51.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.633+0000 7fc933e00640 1 Processor -- start 2024-04-26T21:46:51.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.633+0000 7fc933e00640 1 -- start start 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.634+0000 7fc933e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9340b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.634+0000 7fc933e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc9340b4b30 0x7fc9340d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.634+0000 7fc933e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc934146a60 con 0x7fc9340d0e70 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc933e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc93406e970 con 0x7fc9340b4b30 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc933e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc9340d17f0 con 0x7fc934146680 2024-04-26T21:46:51.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc92a000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc9340d0e70 0x7fc934160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48156/0 (socket says 172.21.15.106:48156) 2024-04-26T21:46:51.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc92a000640 1 -- 172.21.15.106:0/1849547772 learned_addr learned my addr 172.21.15.106:0/1849547772 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:51.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc932a00640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9340b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc933400640 1 --2- 172.21.15.106:0/1849547772 >> v2:172.21.15.204:3300/0 conn(0x7fc9340b4b30 0x7fc9340d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125179758 0 0) 0x7fc934146a60 con 0x7fc9340d0e70 2024-04-26T21:46:51.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.635+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc918003520 con 0x7fc9340d0e70 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3344719260 0 0) 0x7fc918003520 con 0x7fc9340d0e70 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 msgr2=0x7fc9340b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9340b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 >> v2:172.21.15.204:3300/0 conn(0x7fc9340b4b30 msgr2=0x7fc9340d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 --2- 172.21.15.106:0/1849547772 >> v2:172.21.15.204:3300/0 conn(0x7fc9340b4b30 0x7fc9340d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93406dd60 con 0x7fc9340d0e70 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fc91c001670 con 0x7fc9340d0e70 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 >> v1:172.21.15.106:6789/0 conn(0x7fc9340d0e70 legacy=0x7fc934160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9180039b0 0x7fc918003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340d17f0 con 0x7fc918004800 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc91800b9c0 con 0x7fc9180039b0 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc91800bb40 con 0x7fc934146680 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.636+0000 7fc92a000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc932a00640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9180039b0 0x7fc918003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc933400640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc92a000640 1 -- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 msgr2=0x7fc9180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc92a000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc934146680 0x7fc9180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc92a000640 1 -- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9180039b0 msgr2=0x7fc918003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc92a000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9180039b0 0x7fc918003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc92a000640 1 -- 172.21.15.106:0/1849547772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc918003520 con 0x7fc918004800 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.637+0000 7fc932a00640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9180039b0 0x7fc918003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.638+0000 7fc92a000640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc918004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c290 tx=0x7fc918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff260bb6393b0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.638+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007d50 con 0x7fc918004800 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.638+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c005af0 con 0x7fc918004800 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.638+0000 7fc932000640 1 -- 172.21.15.106:0/1849547772 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007070 con 0x7fc918004800 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 -- 172.21.15.106:0/1849547772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 msgr2=0x7fc918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 --2- 172.21.15.106:0/1849547772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc918004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c290 tx=0x7fc918005830 comp rx=0 tx=0).stop 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933400640 1 -- 172.21.15.106:0/1849547772 reap_dead start 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 -- 172.21.15.106:0/1849547772 shutdown_connections 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 -- 172.21.15.106:0/1849547772 >> 172.21.15.106:0/1849547772 conn(0x7fc93406c740 msgr2=0x7fc93406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 -- 172.21.15.106:0/1849547772 shutdown_connections 2024-04-26T21:46:51.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.639+0000 7fc933e00640 1 -- 172.21.15.106:0/1849547772 wait complete. 2024-04-26T21:46:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.641+0000 7fc933e00640 1 Processor -- start 2024-04-26T21:46:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.641+0000 7fc933e00640 1 -- start start 2024-04-26T21:46:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc93413aea0 0x7fc93413b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93413bee0 0x7fc9340732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc93406e600 con 0x7fc918004800 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc93406e320 con 0x7fc93413bee0 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc933e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc9340cd6a0 con 0x7fc93413aea0 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc932a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc932a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53312/0 (socket says 172.21.15.106:53312) 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.642+0000 7fc932a00640 1 -- 172.21.15.106:0/3433163525 learned_addr learned my addr 172.21.15.106:0/3433163525 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 -- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc93413aea0 msgr2=0x7fc93413b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc93413aea0 0x7fc93413b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 -- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93413bee0 msgr2=0x7fc9340732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93413bee0 0x7fc9340732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 -- 172.21.15.106:0/3433163525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc934139e00 con 0x7fc918004800 2024-04-26T21:46:51.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc932a00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc924010890 tx=0x7fc924010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92400f040 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92400ea80 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92400ee50 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93413a0e0 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.643+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93413a6a0 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.645+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9340693c0 con 0x7fc918004800 2024-04-26T21:46:51.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.646+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc92401c400 con 0x7fc918004800 2024-04-26T21:46:51.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.646+0000 7fc92b400640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 0x7fc91006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.647+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc92408c060 con 0x7fc918004800 2024-04-26T21:46:51.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.647+0000 7fc92a000640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 0x7fc91006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.648+0000 7fc92a000640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 0x7fc91006ed20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc91c033000 tx=0x7fc91c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.652+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc924018030 con 0x7fc918004800 2024-04-26T21:46:51.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.850+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9340b6040 con 0x7fc918004800 2024-04-26T21:46:51.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.852+0000 7fc92b400640 1 -- 172.21.15.106:0/3433163525 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7fc924059db0 con 0x7fc918004800 2024-04-26T21:46:51.852 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:51.853 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.854+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 msgr2=0x7fc91006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.854+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 0x7fc91006ed20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc91c033000 tx=0x7fc91c007000 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 msgr2=0x7fc9341457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc924010890 tx=0x7fc924010d50 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 shutdown_connections 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93413bee0 0x7fc9340732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc91006c860 0x7fc91006ed20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc93413aea0 0x7fc93413b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 --2- 172.21.15.106:0/3433163525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc918004800 0x7fc9341457c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 >> 172.21.15.106:0/3433163525 conn(0x7fc93406c740 msgr2=0x7fc93413f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 shutdown_connections 2024-04-26T21:46:51.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:51.855+0000 7fc933e00640 1 -- 172.21.15.106:0/3433163525 wait complete. 2024-04-26T21:46:51.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:51.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.068+0000 7f63bd800640 1 Processor -- start 2024-04-26T21:46:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.069+0000 7f63bd800640 1 -- start start 2024-04-26T21:46:52.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.070+0000 7f63bd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f63b8161e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.070+0000 7f63bd800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b80d3fe0 con 0x7f63b815f2a0 2024-04-26T21:46:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.070+0000 7f63bd800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b8160af0 con 0x7f63b8163890 2024-04-26T21:46:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.070+0000 7f63bd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63b80d0520 con 0x7f63b80d2120 2024-04-26T21:46:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f63b815f2a0 0x7f63b80cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48162/0 (socket says 172.21.15.106:48162) 2024-04-26T21:46:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b7e00640 1 -- 172.21.15.106:0/1340066741 learned_addr learned my addr 172.21.15.106:0/1340066741 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63bce00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f63b8161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827891556 0 0) 0x7f63b80d3fe0 con 0x7f63b815f2a0 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f639c003520 con 0x7f63b815f2a0 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2011513783 0 0) 0x7f639c003520 con 0x7f63b815f2a0 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 msgr2=0x7f63b8161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f63b8161e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.071+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 >> v1:172.21.15.204:6789/0 conn(0x7f63b8163890 legacy=0x7f63b80d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b80d0800 con 0x7f63b815f2a0 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f63a80019f0 con 0x7f63b815f2a0 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 >> v1:172.21.15.106:6789/0 conn(0x7f63b815f2a0 legacy=0x7f63b80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639c003a50 0x7f639c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c004800 0x7f639c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f639c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b80d0520 con 0x7f639c003a50 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f639c00bb20 con 0x7f639c004800 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f639c00bd20 con 0x7f63b80d2120 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639c003a50 0x7f639c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7400640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c004800 0x7f639c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63bce00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f639c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 -- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 msgr2=0x7f639c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63b80d2120 0x7f639c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 -- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c004800 msgr2=0x7f639c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c004800 0x7f639c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7e00640 1 -- 172.21.15.106:0/1340066741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c003520 con 0x7f639c003a50 2024-04-26T21:46:52.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.072+0000 7f63b7400640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c004800 0x7f639c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:52.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63b7e00640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639c003a50 0x7f639c003e70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f63a800a850 tx=0x7f639c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0aa8647b42841a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a800a4a0 con 0x7f639c003a50 2024-04-26T21:46:52.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a802f070 con 0x7f639c003a50 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63b6a00640 1 -- 172.21.15.106:0/1340066741 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a802bc20 con 0x7f639c003a50 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63bd800640 1 -- 172.21.15.106:0/1340066741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639c003a50 msgr2=0x7f639c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63bd800640 1 --2- 172.21.15.106:0/1340066741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639c003a50 0x7f639c003e70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f63a800a850 tx=0x7f639c005830 comp rx=0 tx=0).stop 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63bce00640 1 -- 172.21.15.106:0/1340066741 reap_dead start 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.074+0000 7f63bd800640 1 -- 172.21.15.106:0/1340066741 shutdown_connections 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.075+0000 7f63bd800640 1 -- 172.21.15.106:0/1340066741 >> 172.21.15.106:0/1340066741 conn(0x7f63b809e290 msgr2=0x7f63b80d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.075+0000 7f63bd800640 1 -- 172.21.15.106:0/1340066741 shutdown_connections 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.075+0000 7f63bd800640 1 -- 172.21.15.106:0/1340066741 wait complete. 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.076+0000 7f63bd800640 1 Processor -- start 2024-04-26T21:46:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 -- start start 2024-04-26T21:46:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c003a50 0x7f63b81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 0x7f63b80ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8161e20 con 0x7f63b80b7be0 2024-04-26T21:46:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.077+0000 7f63bd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63b80d0dc0 con 0x7f63b80ce6a0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63bd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63b8161fa0 con 0x7f639c003a50 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53320/0 (socket says 172.21.15.106:53320) 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 -- 172.21.15.106:0/2281746933 learned_addr learned my addr 172.21.15.106:0/2281746933 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 -- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c003a50 msgr2=0x7f63b81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63bce00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 0x7f63b80ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7e00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c003a50 0x7f63b81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c003a50 0x7f63b81598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 -- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 msgr2=0x7f63b80ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 0x7f63b80ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63b7400640 1 -- 172.21.15.106:0/2281746933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b80cd600 con 0x7f63b80b7be0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.078+0000 7f63bce00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 0x7f63b80ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63b7400640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d800 tx=0x7f63a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0005070 con 0x7f63b80b7be0 2024-04-26T21:46:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a0005210 con 0x7f63b80b7be0 2024-04-26T21:46:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0007030 con 0x7f63b80b7be0 2024-04-26T21:46:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b80cd8e0 con 0x7f63b80b7be0 2024-04-26T21:46:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.079+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b80cdea0 con 0x7f63b80b7be0 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.080+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b8069ad0 con 0x7f63b80b7be0 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.081+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63a001a020 con 0x7f63b80b7be0 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.082+0000 7f63afe00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.082+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f63a008b320 con 0x7f63b80b7be0 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.082+0000 7f63b7e00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.083+0000 7f63b7e00640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 0x7f639406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63a8033000 tx=0x7f63a8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.087+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63a0059760 con 0x7f63b80b7be0 2024-04-26T21:46:52.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.286+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f63b80d2740 con 0x7f63b80b7be0 2024-04-26T21:46:52.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.287+0000 7f63afe00640 1 -- 172.21.15.106:0/2281746933 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f63a0059060 con 0x7f63b80b7be0 2024-04-26T21:46:52.287 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 msgr2=0x7f639406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 0x7f639406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63a8033000 tx=0x7f63a8009030 comp rx=0 tx=0).stop 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 msgr2=0x7f63b80b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d800 tx=0x7f63a000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 shutdown_connections 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b80ce6a0 0x7f63b80ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80b7be0 0x7f63b80b8060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 --2- 172.21.15.106:0/2281746933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c003a50 0x7f63b81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 >> 172.21.15.106:0/2281746933 conn(0x7f63b809e290 msgr2=0x7f63b815a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 shutdown_connections 2024-04-26T21:46:52.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.290+0000 7f63bd800640 1 -- 172.21.15.106:0/2281746933 wait complete. 2024-04-26T21:46:52.309 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.498+0000 7fbb95400640 1 Processor -- start 2024-04-26T21:46:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.498+0000 7fbb95400640 1 -- start start 2024-04-26T21:46:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb900b5ca0 0x7fbb9006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb900add80 0x7fbb90072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbb900af800 con 0x7fbb900add80 2024-04-26T21:46:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbb900ae340 con 0x7fbb900b5ca0 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb95400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900afaa0 con 0x7fbb9015d100 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb8f200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb900add80 0x7fbb90072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb8f200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb900add80 0x7fbb90072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53322/0 (socket says 172.21.15.106:53322) 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.499+0000 7fbb8f200640 1 -- 172.21.15.106:0/807780618 learned_addr learned my addr 172.21.15.106:0/807780618 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.500+0000 7fbb8f200640 1 -- 172.21.15.106:0/807780618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 msgr2=0x7fbb9006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.500+0000 7fbb8f200640 1 --2- 172.21.15.106:0/807780618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.500+0000 7fbb8f200640 1 -- 172.21.15.106:0/807780618 >> v2:172.21.15.204:3300/0 conn(0x7fbb900b5ca0 msgr2=0x7fbb9006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.500+0000 7fbb8f200640 1 --2- 172.21.15.106:0/807780618 >> v2:172.21.15.204:3300/0 conn(0x7fbb900b5ca0 0x7fbb9006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.500+0000 7fbb8f200640 1 -- 172.21.15.106:0/807780618 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb900afd80 con 0x7fbb900add80 2024-04-26T21:46:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb8f200640 1 --2- 172.21.15.106:0/807780618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900add80 0x7fbb90072010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb90075d50 tx=0x7fbb8802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623bb0607083c8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb8de00640 1 -- 172.21.15.106:0/807780618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88036070 con 0x7fbb900add80 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb8de00640 1 -- 172.21.15.106:0/807780618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88031600 con 0x7fbb900add80 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb8de00640 1 -- 172.21.15.106:0/807780618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880319d0 con 0x7fbb900add80 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb95400640 1 -- 172.21.15.106:0/807780618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900add80 msgr2=0x7fbb90072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.501+0000 7fbb95400640 1 --2- 172.21.15.106:0/807780618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900add80 0x7fbb90072010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb90075d50 tx=0x7fbb8802a600 comp rx=0 tx=0).stop 2024-04-26T21:46:52.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 -- 172.21.15.106:0/807780618 shutdown_connections 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 --2- 172.21.15.106:0/807780618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900add80 0x7fbb90072010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 --2- 172.21.15.106:0/807780618 >> v2:172.21.15.204:3300/0 conn(0x7fbb900b5ca0 0x7fbb9006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 --2- 172.21.15.106:0/807780618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 -- 172.21.15.106:0/807780618 >> 172.21.15.106:0/807780618 conn(0x7fbb900b2d50 msgr2=0x7fbb900b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 -- 172.21.15.106:0/807780618 shutdown_connections 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.502+0000 7fbb95400640 1 -- 172.21.15.106:0/807780618 wait complete. 2024-04-26T21:46:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.504+0000 7fbb95400640 1 Processor -- start 2024-04-26T21:46:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.504+0000 7fbb95400640 1 -- start start 2024-04-26T21:46:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb900add80 0x7fbb9009c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9009ee50 con 0x7fbb900b5ca0 2024-04-26T21:46:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900b0340 con 0x7fbb900add80 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb95400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb90160b70 con 0x7fbb9015d100 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53328/0 (socket says 172.21.15.106:53328) 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 -- 172.21.15.106:0/1321953177 learned_addr learned my addr 172.21.15.106:0/1321953177 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 -- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 msgr2=0x7fbb9006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 -- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb900add80 msgr2=0x7fbb9009c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb8e800640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb900add80 0x7fbb9009c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb900add80 0x7fbb9009c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.505+0000 7fbb85e00640 1 -- 172.21.15.106:0/1321953177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9009d3b0 con 0x7fbb900b5ca0 2024-04-26T21:46:52.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb85e00640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b4a0 tx=0x7fbb7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78014070 con 0x7fbb900b5ca0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78010610 con 0x7fbb900b5ca0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78010a00 con 0x7fbb900b5ca0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9009d690 con 0x7fbb900b5ca0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.506+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9009dc50 con 0x7fbb900b5ca0 2024-04-26T21:46:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.507+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb90161670 con 0x7fbb900b5ca0 2024-04-26T21:46:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.508+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb78017070 con 0x7fbb900b5ca0 2024-04-26T21:46:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.509+0000 7fbb87400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 0x7fbb6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.509+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbb7808b6c0 con 0x7fbb900b5ca0 2024-04-26T21:46:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.510+0000 7fbb8e800640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 0x7fbb6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.510+0000 7fbb8e800640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 0x7fbb6c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000a770 tx=0x7fbb80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.514+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb90161670 con 0x7fbb900b5ca0 2024-04-26T21:46:52.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.696+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbb90069650 con 0x7fbb900b5ca0 2024-04-26T21:46:52.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.697+0000 7fbb87400640 1 -- 172.21.15.106:0/1321953177 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fbb780599d0 con 0x7fbb900b5ca0 2024-04-26T21:46:52.698 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 msgr2=0x7fbb6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 0x7fbb6c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000a770 tx=0x7fbb80008040 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 msgr2=0x7fbb9006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b4a0 tx=0x7fbb7800b960 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 shutdown_connections 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbb6c06c8f0 0x7fbb6c06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9015d100 0x7fbb9006f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb900b5ca0 0x7fbb9006f110 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 --2- 172.21.15.106:0/1321953177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb900add80 0x7fbb9009c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.700+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 >> 172.21.15.106:0/1321953177 conn(0x7fbb900b2d50 msgr2=0x7fbb900ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.701+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 shutdown_connections 2024-04-26T21:46:52.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.701+0000 7fbb95400640 1 -- 172.21.15.106:0/1321953177 wait complete. 2024-04-26T21:46:52.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.903+0000 7fc3cd800640 1 Processor -- start 2024-04-26T21:46:52.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.904+0000 7fc3cd800640 1 -- start start 2024-04-26T21:46:52.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.905+0000 7fc3cd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 0x7fc3c806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.905+0000 7fc3cd800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c8072e30 con 0x7fc3c80add80 2024-04-26T21:46:52.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.905+0000 7fc3cd800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c80af800 con 0x7fc3c80b5ca0 2024-04-26T21:46:52.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.905+0000 7fc3cd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c80ae340 con 0x7fc3c815d100 2024-04-26T21:46:52.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c7400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc3c80add80 0x7fc3c8072720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48172/0 (socket says 172.21.15.106:48172) 2024-04-26T21:46:52.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c7400640 1 -- 172.21.15.106:0/2798534631 learned_addr learned my addr 172.21.15.106:0/2798534631 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:52.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6a00640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 0x7fc3c806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 381598862 0 0) 0x7fc3c8072e30 con 0x7fc3c80add80 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3ac003520 con 0x7fc3c80add80 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 445188356 0 0) 0x7fc3ac003520 con 0x7fc3c80add80 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 msgr2=0x7fc3c806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 0x7fc3c806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 >> v1:172.21.15.204:6789/0 conn(0x7fc3c80b5ca0 legacy=0x7fc3c8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.906+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c80afaa0 con 0x7fc3c80add80 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fc3c0001670 con 0x7fc3c80add80 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 >> v1:172.21.15.106:6789/0 conn(0x7fc3c80add80 legacy=0x7fc3c8072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac0039e0 0x7fc3ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3ac004800 0x7fc3ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 0x7fc3ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80ae340 con 0x7fc3ac004800 2024-04-26T21:46:52.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac00baf0 con 0x7fc3ac0039e0 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac00bcf0 con 0x7fc3c815d100 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3ac004800 0x7fc3ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3c6a00640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac0039e0 0x7fc3ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 -- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 msgr2=0x7fc3ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c815d100 0x7fc3ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 -- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac0039e0 msgr2=0x7fc3ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac0039e0 0x7fc3ac003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.907+0000 7fc3be000640 1 -- 172.21.15.106:0/2798534631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac003520 con 0x7fc3ac004800 2024-04-26T21:46:52.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.908+0000 7fc3be000640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3ac004800 0x7fc3ac004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001750 tx=0x7fc3ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40955f991b4f09b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0001370 con 0x7fc3ac004800 2024-04-26T21:46:52.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b00052d0 con 0x7fc3ac004800 2024-04-26T21:46:52.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3c6000640 1 -- 172.21.15.106:0/2798534631 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b002abe0 con 0x7fc3ac004800 2024-04-26T21:46:52.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3cd800640 1 -- 172.21.15.106:0/2798534631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3ac004800 msgr2=0x7fc3ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3cd800640 1 --2- 172.21.15.106:0/2798534631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3ac004800 0x7fc3ac004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001750 tx=0x7fc3ac005830 comp rx=0 tx=0).stop 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3c7400640 1 -- 172.21.15.106:0/2798534631 reap_dead start 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3cd800640 1 -- 172.21.15.106:0/2798534631 shutdown_connections 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3cd800640 1 -- 172.21.15.106:0/2798534631 >> 172.21.15.106:0/2798534631 conn(0x7fc3c80b2d50 msgr2=0x7fc3c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.909+0000 7fc3cd800640 1 -- 172.21.15.106:0/2798534631 shutdown_connections 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.910+0000 7fc3cd800640 1 -- 172.21.15.106:0/2798534631 wait complete. 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.911+0000 7fc3cd800640 1 Processor -- start 2024-04-26T21:46:52.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.911+0000 7fc3cd800640 1 -- start start 2024-04-26T21:46:52.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac004800 0x7fc3c814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c806f1f0 0x7fc3c806f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80b0340 con 0x7fc3c809dc20 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80b0060 con 0x7fc3ac004800 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.912+0000 7fc3cd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c809ee50 con 0x7fc3c806f1f0 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53346/0 (socket says 172.21.15.106:53346) 2024-04-26T21:46:52.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 -- 172.21.15.106:0/1487164345 learned_addr learned my addr 172.21.15.106:0/1487164345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c6a00640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac004800 0x7fc3c814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 -- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c806f1f0 msgr2=0x7fc3c806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c806f1f0 0x7fc3c806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 -- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac004800 msgr2=0x7fc3c814d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac004800 0x7fc3c814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 -- 172.21.15.106:0/1487164345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c806e380 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3c7400640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80737f0 tx=0x7fc3c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.913+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00074e0 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.914+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c00047e0 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.914+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0004bd0 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.914+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c806e660 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.914+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c806ec20 con 0x7fc3c809dc20 2024-04-26T21:46:52.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.915+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c8069650 con 0x7fc3c809dc20 2024-04-26T21:46:52.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.916+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3c0011070 con 0x7fc3c809dc20 2024-04-26T21:46:52.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.917+0000 7fc3bf400640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 0x7fc3a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.917+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc3c008b270 con 0x7fc3c809dc20 2024-04-26T21:46:52.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.918+0000 7fc3c6a00640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 0x7fc3a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.918+0000 7fc3c6a00640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 0x7fc3a406ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8003a50 tx=0x7fc3b800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:52.922+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3c0059610 con 0x7fc3c809dc20 2024-04-26T21:46:53.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.111+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3c80d1ba0 con 0x7fc3c809dc20 2024-04-26T21:46:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.112+0000 7fc3bf400640 1 -- 172.21.15.106:0/1487164345 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc3c0058f10 con 0x7fc3c809dc20 2024-04-26T21:46:53.113 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.114+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 msgr2=0x7fc3a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.114+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 0x7fc3a406ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8003a50 tx=0x7fc3b800b040 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 msgr2=0x7fc3c809e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80737f0 tx=0x7fc3c000c960 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 shutdown_connections 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3c809dc20 0x7fc3c809e0a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc3a406c920 0x7fc3a406ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3c806f1f0 0x7fc3c806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 --2- 172.21.15.106:0/1487164345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac004800 0x7fc3c814d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 >> 172.21.15.106:0/1487164345 conn(0x7fc3c80b2d50 msgr2=0x7fc3c80b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 shutdown_connections 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.115+0000 7fc3cd800640 1 -- 172.21.15.106:0/1487164345 wait complete. 2024-04-26T21:46:53.133 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:53.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.324+0000 7f60df200640 1 Processor -- start 2024-04-26T21:46:53.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.324+0000 7f60df200640 1 -- start start 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60df200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60e0146680 0x7f60e00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60df200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 0x7f60e00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60df200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e0146a60 con 0x7f60e0160930 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60df200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60e006e970 con 0x7f60e0146680 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60df200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f60e00d17f0 con 0x7f60e00d0e70 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60de800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f60e0160930 0x7f60e0160d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48182/0 (socket says 172.21.15.106:48182) 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60de800640 1 -- 172.21.15.106:0/2951829492 learned_addr learned my addr 172.21.15.106:0/2951829492 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:53.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60dde00640 1 --2- 172.21.15.106:0/2951829492 >> v2:172.21.15.106:3300/0 conn(0x7f60e0146680 0x7f60e00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60dca00640 1 -- 172.21.15.106:0/2951829492 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934604810 0 0) 0x7f60e0146a60 con 0x7f60e0160930 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60dd400640 1 --2- 172.21.15.106:0/2951829492 >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 0x7f60e00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.326+0000 7f60dca00640 1 -- 172.21.15.106:0/2951829492 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c4003520 con 0x7f60e0160930 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.327+0000 7f60dde00640 1 -- 172.21.15.106:0/2951829492 >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 msgr2=0x7f60e00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.327+0000 7f60dde00640 1 --2- 172.21.15.106:0/2951829492 >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 0x7f60e00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.327+0000 7f60dde00640 1 -- 172.21.15.106:0/2951829492 >> v1:172.21.15.106:6789/0 conn(0x7f60e0160930 legacy=0x7f60e0160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.327+0000 7f60dde00640 1 -- 172.21.15.106:0/2951829492 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e006dd60 con 0x7f60e0146680 2024-04-26T21:46:53.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.327+0000 7f60dd400640 1 --2- 172.21.15.106:0/2951829492 >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 0x7f60e00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:53.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60dde00640 1 --2- 172.21.15.106:0/2951829492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e00b5990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f60e0160890 tx=0x7f60d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc9892b53743e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60dca00640 1 -- 172.21.15.106:0/2951829492 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0037070 con 0x7f60e0146680 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60dca00640 1 -- 172.21.15.106:0/2951829492 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d0032440 con 0x7f60e0146680 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60dca00640 1 -- 172.21.15.106:0/2951829492 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d00327f0 con 0x7f60e0146680 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60df200640 1 -- 172.21.15.106:0/2951829492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 msgr2=0x7f60e00b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60df200640 1 --2- 172.21.15.106:0/2951829492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e00b5990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f60e0160890 tx=0x7f60d002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.328+0000 7f60df200640 1 -- 172.21.15.106:0/2951829492 shutdown_connections 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.329+0000 7f60df200640 1 --2- 172.21.15.106:0/2951829492 >> v2:172.21.15.204:3300/0 conn(0x7f60e00d0e70 0x7f60e00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.329+0000 7f60df200640 1 --2- 172.21.15.106:0/2951829492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e00b5990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.329+0000 7f60df200640 1 -- 172.21.15.106:0/2951829492 >> 172.21.15.106:0/2951829492 conn(0x7f60e006c740 msgr2=0x7f60e006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.329+0000 7f60df200640 1 -- 172.21.15.106:0/2951829492 shutdown_connections 2024-04-26T21:46:53.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.329+0000 7f60df200640 1 -- 172.21.15.106:0/2951829492 wait complete. 2024-04-26T21:46:53.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.330+0000 7f60df200640 1 Processor -- start 2024-04-26T21:46:53.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.330+0000 7f60df200640 1 -- start start 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e00d0e70 0x7f60e015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60e0160930 0x7f60e0143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60e006e600 con 0x7f60e0146680 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60e006e320 con 0x7f60e00d0e70 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.331+0000 7f60df200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60e00cd6a0 con 0x7f60e0160930 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53360/0 (socket says 172.21.15.106:53360) 2024-04-26T21:46:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 -- 172.21.15.106:0/119835424 learned_addr learned my addr 172.21.15.106:0/119835424 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dde00640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e00d0e70 0x7f60e015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 -- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60e0160930 msgr2=0x7f60e0143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60e0160930 0x7f60e0143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 -- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e00d0e70 msgr2=0x7f60e015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e00d0e70 0x7f60e015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 -- 172.21.15.106:0/119835424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e013b910 con 0x7f60e0146680 2024-04-26T21:46:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.332+0000 7f60dd400640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c7f0 tx=0x7f60cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.333+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc015070 con 0x7f60e0146680 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.333+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc007a30 con 0x7f60e0146680 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.333+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc007e00 con 0x7f60e0146680 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.333+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e013bbf0 con 0x7f60e0146680 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.333+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e013c1b0 con 0x7f60e0146680 2024-04-26T21:46:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.334+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e00693c0 con 0x7f60e0146680 2024-04-26T21:46:53.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.335+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60cc018070 con 0x7f60e0146680 2024-04-26T21:46:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.336+0000 7f60d6000640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.336+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f60cc08bce0 con 0x7f60e0146680 2024-04-26T21:46:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.336+0000 7f60dde00640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.337+0000 7f60dde00640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 0x7f60c006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60d0034000 tx=0x7f60d002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.340+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60cc059f60 con 0x7f60e0146680 2024-04-26T21:46:53.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.552+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60e0141820 con 0x7f60e0146680 2024-04-26T21:46:53.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.553+0000 7f60d6000640 1 -- 172.21.15.106:0/119835424 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f60cc059860 con 0x7f60e0146680 2024-04-26T21:46:53.554 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.555+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 msgr2=0x7f60c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.555+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 0x7f60c006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60d0034000 tx=0x7f60d002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 msgr2=0x7f60e013b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c7f0 tx=0x7f60cc00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 shutdown_connections 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60e0160930 0x7f60e0143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60e0146680 0x7f60e013b0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 --2- 172.21.15.106:0/119835424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e00d0e70 0x7f60e015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 >> 172.21.15.106:0/119835424 conn(0x7f60e006c740 msgr2=0x7f60e015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 shutdown_connections 2024-04-26T21:46:53.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:53.556+0000 7f60df200640 1 -- 172.21.15.106:0/119835424 wait complete. 2024-04-26T21:46:53.574 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-26T21:46:53.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:53.575 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-26T21:46:53.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:46:53.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:46:53.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:53.655 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T21:46:53.709 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:53.709 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:53.765 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:53.766 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:53.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:54.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.028+0000 7fb03ea00640 1 Processor -- start 2024-04-26T21:46:54.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.028+0000 7fb03ea00640 1 -- start start 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb038146680 0x7fb0380b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03ea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb038146a60 con 0x7fb0380d0e70 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03ea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb038161750 con 0x7fb0380b4b30 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03ea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb03806e970 con 0x7fb038146680 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03cc00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb0380d0e70 0x7fb038160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48192/0 (socket says 172.21.15.106:48192) 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.030+0000 7fb03cc00640 1 -- 172.21.15.106:0/3026272507 learned_addr learned my addr 172.21.15.106:0/3026272507 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb03d600640 1 --2- 172.21.15.106:0/3026272507 >> v2:172.21.15.204:3300/0 conn(0x7fb038146680 0x7fb0380b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106949769 0 0) 0x7fb038146a60 con 0x7fb0380d0e70 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb020003520 con 0x7fb0380d0e70 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 18034495 0 0) 0x7fb020003520 con 0x7fb0380d0e70 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 >> v2:172.21.15.204:3300/0 conn(0x7fb038146680 msgr2=0x7fb0380b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 --2- 172.21.15.106:0/3026272507 >> v2:172.21.15.204:3300/0 conn(0x7fb038146680 0x7fb0380b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 >> v1:172.21.15.204:6789/0 conn(0x7fb0380b4b30 legacy=0x7fb038161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03806dd60 con 0x7fb0380d0e70 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fb024001670 con 0x7fb0380d0e70 2024-04-26T21:46:54.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 >> v1:172.21.15.106:6789/0 conn(0x7fb0380d0e70 legacy=0x7fb038160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0200039b0 0x7fb020003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 0x7fb020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0200051c0 0x7fb02000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb02000c0c0 con 0x7fb020004800 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb02000c270 con 0x7fb0200039b0 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.031+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb02000c470 con 0x7fb0200051c0 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 0x7fb020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03d600640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0200039b0 0x7fb020003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03e000640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0200051c0 0x7fb02000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 -- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0200051c0 msgr2=0x7fb02000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0200051c0 0x7fb02000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 -- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0200039b0 msgr2=0x7fb020003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0200039b0 0x7fb020003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03e000640 1 -- 172.21.15.106:0/3026272507 reap_dead start 2024-04-26T21:46:54.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.032+0000 7fb03cc00640 1 -- 172.21.15.106:0/3026272507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020003520 con 0x7fb020004800 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.034+0000 7fb03cc00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 0x7fb020004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c210 tx=0x7fb020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5171e9dd559b4cab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.034+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024007d10 con 0x7fb020004800 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.034+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb024005ad0 con 0x7fb020004800 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.034+0000 7fb037e00640 1 -- 172.21.15.106:0/3026272507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024007070 con 0x7fb020004800 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 -- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 msgr2=0x7fb020004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 0x7fb020004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c210 tx=0x7fb020005830 comp rx=0 tx=0).stop 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 -- 172.21.15.106:0/3026272507 shutdown_connections 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 --2- 172.21.15.106:0/3026272507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020004800 0x7fb020004c80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 -- 172.21.15.106:0/3026272507 >> 172.21.15.106:0/3026272507 conn(0x7fb03806c740 msgr2=0x7fb03806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 -- 172.21.15.106:0/3026272507 shutdown_connections 2024-04-26T21:46:54.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.035+0000 7fb03ea00640 1 -- 172.21.15.106:0/3026272507 wait complete. 2024-04-26T21:46:54.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.037+0000 7fb03ea00640 1 Processor -- start 2024-04-26T21:46:54.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.037+0000 7fb03ea00640 1 -- start start 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb020004800 0x7fb0380ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb03815c5d0 0x7fb038154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb03806e600 con 0x7fb0380cc360 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb03806e320 con 0x7fb020004800 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb0380cd6a0 con 0x7fb03815c5d0 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53364/0 (socket says 172.21.15.106:53364) 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 -- 172.21.15.106:0/1097647474 learned_addr learned my addr 172.21.15.106:0/1097647474 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03d600640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb020004800 0x7fb0380ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03cc00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb03815c5d0 0x7fb038154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 -- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb03815c5d0 msgr2=0x7fb038154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb03815c5d0 0x7fb038154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 -- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb020004800 msgr2=0x7fb0380ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.038+0000 7fb03e000640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb020004800 0x7fb0380ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb03e000640 1 -- 172.21.15.106:0/1097647474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03813c1e0 con 0x7fb0380cc360 2024-04-26T21:46:54.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb03e000640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0380d1b00 tx=0x7fb03000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03001b070 con 0x7fb0380cc360 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb030003df0 con 0x7fb0380cc360 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030013640 con 0x7fb0380cc360 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03813c4c0 con 0x7fb0380cc360 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.039+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0380ccd20 con 0x7fb0380cc360 2024-04-26T21:46:54.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.040+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0380cd240 con 0x7fb0380cc360 2024-04-26T21:46:54.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.041+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb03001e070 con 0x7fb0380cc360 2024-04-26T21:46:54.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.042+0000 7fb035600640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 0x7fb01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.042+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb03008b550 con 0x7fb0380cc360 2024-04-26T21:46:54.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.043+0000 7fb03d600640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 0x7fb01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.043+0000 7fb03d600640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 0x7fb01806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb038051930 tx=0x7fb02c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.047+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb030059860 con 0x7fb0380cc360 2024-04-26T21:46:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.248+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0380693c0 con 0x7fb0380cc360 2024-04-26T21:46:54.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.251+0000 7fb035600640 1 -- 172.21.15.106:0/1097647474 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7fb030059160 con 0x7fb0380cc360 2024-04-26T21:46:54.252 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:54.252 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.253+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 msgr2=0x7fb01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.253+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 0x7fb01806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb038051930 tx=0x7fb02c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 msgr2=0x7fb0380cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0380d1b00 tx=0x7fb03000ea00 comp rx=0 tx=0).stop 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 shutdown_connections 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0380cc360 0x7fb0380cc7e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb01806c8f0 0x7fb01806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb03815c5d0 0x7fb038154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 --2- 172.21.15.106:0/1097647474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb020004800 0x7fb0380ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 >> 172.21.15.106:0/1097647474 conn(0x7fb03806c740 msgr2=0x7fb0380cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 shutdown_connections 2024-04-26T21:46:54.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.254+0000 7fb03ea00640 1 -- 172.21.15.106:0/1097647474 wait complete. 2024-04-26T21:46:54.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:54.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:54.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.476+0000 7fe688600640 1 Processor -- start 2024-04-26T21:46:54.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.477+0000 7fe688600640 1 -- start start 2024-04-26T21:46:54.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.478+0000 7fe688600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.478+0000 7fe688600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe680071fe0 0x7fe68009bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.478+0000 7fe688600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68009c6f0 con 0x7fe680070420 2024-04-26T21:46:54.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.478+0000 7fe688600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe6800d1ad0 con 0x7fe680071fe0 2024-04-26T21:46:54.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.478+0000 7fe688600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6800b5a60 con 0x7fe680160a90 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:55998/0 (socket says 172.21.15.106:55998) 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 -- 172.21.15.106:0/1224685101 learned_addr learned my addr 172.21.15.106:0/1224685101 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685200640 1 --2- 172.21.15.106:0/1224685101 >> v2:172.21.15.204:3300/0 conn(0x7fe680071fe0 0x7fe68009bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 -- 172.21.15.106:0/1224685101 >> v2:172.21.15.204:3300/0 conn(0x7fe680071fe0 msgr2=0x7fe68009bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 --2- 172.21.15.106:0/1224685101 >> v2:172.21.15.204:3300/0 conn(0x7fe680071fe0 0x7fe68009bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 -- 172.21.15.106:0/1224685101 >> v1:172.21.15.204:6789/0 conn(0x7fe680070420 legacy=0x7fe680070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.479+0000 7fe685c00640 1 -- 172.21.15.106:0/1224685101 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6800d11a0 con 0x7fe680160a90 2024-04-26T21:46:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.481+0000 7fe685c00640 1 --2- 172.21.15.106:0/1224685101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe670009e50 tx=0x7fe67002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60a449e6017b93c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.481+0000 7fe67fe00640 1 -- 172.21.15.106:0/1224685101 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670036070 con 0x7fe680160a90 2024-04-26T21:46:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.481+0000 7fe67fe00640 1 -- 172.21.15.106:0/1224685101 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670031600 con 0x7fe680160a90 2024-04-26T21:46:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.481+0000 7fe67fe00640 1 -- 172.21.15.106:0/1224685101 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6700319d0 con 0x7fe680160a90 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 -- 172.21.15.106:0/1224685101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 msgr2=0x7fe68009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 --2- 172.21.15.106:0/1224685101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe670009e50 tx=0x7fe67002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 -- 172.21.15.106:0/1224685101 shutdown_connections 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 --2- 172.21.15.106:0/1224685101 >> v2:172.21.15.204:3300/0 conn(0x7fe680071fe0 0x7fe68009bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 --2- 172.21.15.106:0/1224685101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe68009ba50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 -- 172.21.15.106:0/1224685101 >> 172.21.15.106:0/1224685101 conn(0x7fe6800b4bf0 msgr2=0x7fe6800b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 -- 172.21.15.106:0/1224685101 shutdown_connections 2024-04-26T21:46:54.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.482+0000 7fe688600640 1 -- 172.21.15.106:0/1224685101 wait complete. 2024-04-26T21:46:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.484+0000 7fe688600640 1 Processor -- start 2024-04-26T21:46:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.484+0000 7fe688600640 1 -- start start 2024-04-26T21:46:54.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 0x7fe6800cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe680154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800b5710 con 0x7fe680070420 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800d1760 con 0x7fe680071fe0 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe688600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6800b2260 con 0x7fe680160a90 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe685c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe685c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53380/0 (socket says 172.21.15.106:53380) 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.485+0000 7fe685c00640 1 -- 172.21.15.106:0/2280081846 learned_addr learned my addr 172.21.15.106:0/2280081846 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685200640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 0x7fe6800cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 -- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 msgr2=0x7fe680154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe686600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe680154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe680154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 -- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 msgr2=0x7fe6800cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 0x7fe6800cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 -- 172.21.15.106:0/2280081846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe680156d70 con 0x7fe680070420 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685200640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 0x7fe6800cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:54.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe686600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe680154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe685c00640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe670000f80 tx=0x7fe670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670041020 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670007820 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6800ccfc0 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.486+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670036040 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.487+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe680157300 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.488+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe680069ce0 con 0x7fe680070420 2024-04-26T21:46:54.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.489+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe670044020 con 0x7fe680070420 2024-04-26T21:46:54.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.490+0000 7fe67d600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 0x7fe66006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.490+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe6700ac910 con 0x7fe680070420 2024-04-26T21:46:54.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.490+0000 7fe685200640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 0x7fe66006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.491+0000 7fe685200640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 0x7fe66006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe6800b8dc0 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.494+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe680069ce0 con 0x7fe680070420 2024-04-26T21:46:54.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.694+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe680069650 con 0x7fe680070420 2024-04-26T21:46:54.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.694+0000 7fe67d600640 1 -- 172.21.15.106:0/2280081846 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe67004b030 con 0x7fe680070420 2024-04-26T21:46:54.696 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 msgr2=0x7fe66006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 0x7fe66006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe6800b8dc0 tx=0x7fe66c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 msgr2=0x7fe6800cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe670000f80 tx=0x7fe670004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 shutdown_connections 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe66006c8f0 0x7fe66006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe680160a90 0x7fe680154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe680071fe0 0x7fe6800cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 --2- 172.21.15.106:0/2280081846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe680070420 0x7fe6800cc970 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.697+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 >> 172.21.15.106:0/2280081846 conn(0x7fe6800b4bf0 msgr2=0x7fe68015fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.698+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 shutdown_connections 2024-04-26T21:46:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.698+0000 7fe688600640 1 -- 172.21.15.106:0/2280081846 wait complete. 2024-04-26T21:46:54.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:54.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.919+0000 7fd053e00640 1 Processor -- start 2024-04-26T21:46:54.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.919+0000 7fd053e00640 1 -- start start 2024-04-26T21:46:54.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd054160d60 0x7fd0540b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd054161140 0x7fd05406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054146280 0x7fd0541466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd054146c30 con 0x7fd054160d60 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd0540b4d20 con 0x7fd054161140 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.920+0000 7fd053e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0540b5000 con 0x7fd054146280 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd054160d60 0x7fd0540b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd054160d60 0x7fd0540b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53396/0 (socket says 172.21.15.106:53396) 2024-04-26T21:46:54.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 -- 172.21.15.106:0/326466829 learned_addr learned my addr 172.21.15.106:0/326466829 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052000640 1 --2- 172.21.15.106:0/326466829 >> v2:172.21.15.204:3300/0 conn(0x7fd054161140 0x7fd05406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd053400640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054146280 0x7fd0541466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 -- 172.21.15.106:0/326466829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054146280 msgr2=0x7fd0541466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054146280 0x7fd0541466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 -- 172.21.15.106:0/326466829 >> v2:172.21.15.204:3300/0 conn(0x7fd054161140 msgr2=0x7fd05406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 --2- 172.21.15.106:0/326466829 >> v2:172.21.15.204:3300/0 conn(0x7fd054161140 0x7fd05406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.921+0000 7fd052a00640 1 -- 172.21.15.106:0/326466829 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540b52e0 con 0x7fd054160d60 2024-04-26T21:46:54.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.923+0000 7fd052a00640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054160d60 0x7fd0540b5b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd044009870 tx=0x7fd04402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd8f9355e3fcfcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.923+0000 7fd051600640 1 -- 172.21.15.106:0/326466829 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044037070 con 0x7fd054160d60 2024-04-26T21:46:54.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.923+0000 7fd051600640 1 -- 172.21.15.106:0/326466829 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044032440 con 0x7fd054160d60 2024-04-26T21:46:54.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.923+0000 7fd051600640 1 -- 172.21.15.106:0/326466829 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044032810 con 0x7fd054160d60 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 -- 172.21.15.106:0/326466829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054160d60 msgr2=0x7fd0540b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054160d60 0x7fd0540b5b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd044009870 tx=0x7fd04402a210 comp rx=0 tx=0).stop 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 -- 172.21.15.106:0/326466829 shutdown_connections 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054146280 0x7fd0541466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 --2- 172.21.15.106:0/326466829 >> v2:172.21.15.204:3300/0 conn(0x7fd054161140 0x7fd05406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 --2- 172.21.15.106:0/326466829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054160d60 0x7fd0540b5b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 -- 172.21.15.106:0/326466829 >> 172.21.15.106:0/326466829 conn(0x7fd0540b2900 msgr2=0x7fd0540b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 -- 172.21.15.106:0/326466829 shutdown_connections 2024-04-26T21:46:54.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.924+0000 7fd053e00640 1 -- 172.21.15.106:0/326466829 wait complete. 2024-04-26T21:46:54.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.926+0000 7fd053e00640 1 Processor -- start 2024-04-26T21:46:54.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.926+0000 7fd053e00640 1 -- start start 2024-04-26T21:46:54.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054160d60 0x7fd05415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 0x7fd054141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0540b6070 con 0x7fd054146280 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0540cf6f0 con 0x7fd054161140 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd05406e420 con 0x7fd054160d60 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd052a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd052a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53400/0 (socket says 172.21.15.106:53400) 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd052a00640 1 -- 172.21.15.106:0/1625495875 learned_addr learned my addr 172.21.15.106:0/1625495875 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd052000640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054160d60 0x7fd05415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd053400640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 0x7fd054141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.927+0000 7fd052a00640 1 -- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054160d60 msgr2=0x7fd05415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd052a00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054160d60 0x7fd05415bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd052a00640 1 -- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 msgr2=0x7fd054141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd052a00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 0x7fd054141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd052a00640 1 -- 172.21.15.106:0/1625495875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05415b1c0 con 0x7fd054146280 2024-04-26T21:46:54.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd053400640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 0x7fd054141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd052a00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd044000c00 tx=0x7fd0440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044041020 con 0x7fd054146280 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0440306c0 con 0x7fd054146280 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.928+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044037040 con 0x7fd054146280 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.929+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05415b4a0 con 0x7fd054146280 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.929+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05415dc10 con 0x7fd054146280 2024-04-26T21:46:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.930+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0540693c0 con 0x7fd054146280 2024-04-26T21:46:54.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.931+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd044044020 con 0x7fd054146280 2024-04-26T21:46:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.932+0000 7fd04b400640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 0x7fd03006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.932+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd0440ac430 con 0x7fd054146280 2024-04-26T21:46:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.932+0000 7fd052000640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 0x7fd03006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.933+0000 7fd052000640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 0x7fd03006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:54.937+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd04404b030 con 0x7fd054146280 2024-04-26T21:46:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.130+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd05406f2f0 con 0x7fd054146280 2024-04-26T21:46:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.130+0000 7fd04b400640 1 -- 172.21.15.106:0/1625495875 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fd04407a7d0 con 0x7fd054146280 2024-04-26T21:46:55.132 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 msgr2=0x7fd03006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 0x7fd03006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 msgr2=0x7fd0541597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd044000c00 tx=0x7fd0440046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 shutdown_connections 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054161140 0x7fd054141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd03006c8f0 0x7fd03006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054160d60 0x7fd05415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 --2- 172.21.15.106:0/1625495875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054146280 0x7fd0541597c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 >> 172.21.15.106:0/1625495875 conn(0x7fd0540b2900 msgr2=0x7fd0540b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 shutdown_connections 2024-04-26T21:46:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.133+0000 7fd053e00640 1 -- 172.21.15.106:0/1625495875 wait complete. 2024-04-26T21:46:55.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:55.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.352+0000 7f759dc00640 1 Processor -- start 2024-04-26T21:46:55.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.353+0000 7f759dc00640 1 -- start start 2024-04-26T21:46:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.354+0000 7f759dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7598156650 0x7f759815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.354+0000 7f759dc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f759806c770 0x7f759806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.354+0000 7f759dc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759806eea0 con 0x7f7598073120 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.354+0000 7f759dc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f75980b4c40 con 0x7f759806c770 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.354+0000 7f759dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f759806f140 con 0x7f7598156650 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f759806c770 0x7f759806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7597400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7598073120 0x7f75980735b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48200/0 (socket says 172.21.15.106:48200) 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7597400640 1 -- 172.21.15.106:0/2854382115 learned_addr learned my addr 172.21.15.106:0/2854382115 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596a00640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7598156650 0x7f759815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596000640 1 -- 172.21.15.106:0/2854382115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7598156650 msgr2=0x7f759815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596000640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7598156650 0x7f759815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596000640 1 -- 172.21.15.106:0/2854382115 >> v1:172.21.15.106:6789/0 conn(0x7f7598073120 legacy=0x7f75980735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.355+0000 7f7596000640 1 -- 172.21.15.106:0/2854382115 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759806f420 con 0x7f759806c770 2024-04-26T21:46:55.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.356+0000 7f7596000640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759806c770 0x7f759806cbf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75800097a0 tx=0x7f758002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e7c6ba7bb1898a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f7595600640 1 -- 172.21.15.106:0/2854382115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580037070 con 0x7f759806c770 2024-04-26T21:46:55.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f7595600640 1 -- 172.21.15.106:0/2854382115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7580032440 con 0x7f759806c770 2024-04-26T21:46:55.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f7595600640 1 -- 172.21.15.106:0/2854382115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580032810 con 0x7f759806c770 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 -- 172.21.15.106:0/2854382115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759806c770 msgr2=0x7f759806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759806c770 0x7f759806cbf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75800097a0 tx=0x7f758002a180 comp rx=0 tx=0).stop 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 -- 172.21.15.106:0/2854382115 shutdown_connections 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759806c770 0x7f759806cbf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 --2- 172.21.15.106:0/2854382115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7598156650 0x7f759815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 -- 172.21.15.106:0/2854382115 >> 172.21.15.106:0/2854382115 conn(0x7f75980b2d50 msgr2=0x7f75980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 -- 172.21.15.106:0/2854382115 shutdown_connections 2024-04-26T21:46:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.357+0000 7f759dc00640 1 -- 172.21.15.106:0/2854382115 wait complete. 2024-04-26T21:46:55.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.359+0000 7f759dc00640 1 Processor -- start 2024-04-26T21:46:55.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.359+0000 7f759dc00640 1 -- start start 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759806c770 0x7f7598147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7598073120 0x7f7598157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f759806fcc0 con 0x7f7598156650 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f759806f9e0 con 0x7f7598073120 2024-04-26T21:46:55.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.360+0000 7f759dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f759815d320 con 0x7f759806c770 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7596a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759806c770 0x7f7598147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53404/0 (socket says 172.21.15.106:53404) 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 -- 172.21.15.106:0/2853904838 learned_addr learned my addr 172.21.15.106:0/2853904838 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7596000640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7598073120 0x7f7598157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 -- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759806c770 msgr2=0x7f7598147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759806c770 0x7f7598147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 -- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7598073120 msgr2=0x7f7598157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7598073120 0x7f7598157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 -- 172.21.15.106:0/2853904838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7598157980 con 0x7f7598156650 2024-04-26T21:46:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.361+0000 7f7597400640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f759000cbc0 tx=0x7f759000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.362+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590017070 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.362+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759000ad40 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.362+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590012600 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.362+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7598158670 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.362+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7598158c30 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.363+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7598069650 con 0x7f7598156650 2024-04-26T21:46:55.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.364+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f759001a070 con 0x7f7598156650 2024-04-26T21:46:55.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.365+0000 7f758ea00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 0x7f757406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.365+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f759008b850 con 0x7f7598156650 2024-04-26T21:46:55.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.365+0000 7f7596a00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 0x7f757406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.366+0000 7f7596a00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 0x7f757406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f75880079a0 tx=0x7f7588008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.369+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7598157980 con 0x7f7598156650 2024-04-26T21:46:55.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.567+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f759815cbc0 con 0x7f7598156650 2024-04-26T21:46:55.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.567+0000 7f758ea00640 1 -- 172.21.15.106:0/2853904838 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f75900595a0 con 0x7f7598156650 2024-04-26T21:46:55.568 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:55.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 msgr2=0x7f757406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 0x7f757406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f75880079a0 tx=0x7f7588008040 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 msgr2=0x7f75980af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f759000cbc0 tx=0x7f759000a430 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 shutdown_connections 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7598156650 0x7f75980af750 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7598073120 0x7f7598157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f757406c920 0x7f757406ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 --2- 172.21.15.106:0/2853904838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759806c770 0x7f7598147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.570+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 >> 172.21.15.106:0/2853904838 conn(0x7f75980b2d50 msgr2=0x7f75980b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.571+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 shutdown_connections 2024-04-26T21:46:55.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.571+0000 7f759dc00640 1 -- 172.21.15.106:0/2853904838 wait complete. 2024-04-26T21:46:55.592 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:55.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.788+0000 7f109da00640 1 Processor -- start 2024-04-26T21:46:55.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.788+0000 7f109da00640 1 -- start start 2024-04-26T21:46:55.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.789+0000 7f109da00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10980cd650 0x7f10980add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.789+0000 7f109da00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10980af9a0 con 0x7f109806ce80 2024-04-26T21:46:55.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.789+0000 7f109da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10980afc80 con 0x7f109815cdf0 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.789+0000 7f109da00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10980ae2c0 con 0x7f10980cd650 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1097400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10980cd650 0x7f10980add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1096000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f109806ce80 0x7f10980b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48216/0 (socket says 172.21.15.106:48216) 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1096000640 1 -- 172.21.15.106:0/3028761000 learned_addr learned my addr 172.21.15.106:0/3028761000 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1097400640 1 -- 172.21.15.106:0/3028761000 >> v1:172.21.15.204:6789/0 conn(0x7f109815cdf0 legacy=0x7f109806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:55.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1097400640 1 -- 172.21.15.106:0/3028761000 >> v1:172.21.15.106:6789/0 conn(0x7f109806ce80 legacy=0x7f10980b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.790+0000 7f1097400640 1 -- 172.21.15.106:0/3028761000 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10980aff60 con 0x7f10980cd650 2024-04-26T21:46:55.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.791+0000 7f1097400640 1 --2- 172.21.15.106:0/3028761000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10980cd650 0x7f10980add80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f109002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4e4000cd6277cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.791+0000 7f1095600640 1 -- 172.21.15.106:0/3028761000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090007740 con 0x7f10980cd650 2024-04-26T21:46:55.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f1095600640 1 -- 172.21.15.106:0/3028761000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10900078e0 con 0x7f10980cd650 2024-04-26T21:46:55.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f1095600640 1 -- 172.21.15.106:0/3028761000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090007cb0 con 0x7f10980cd650 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 -- 172.21.15.106:0/3028761000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10980cd650 msgr2=0x7f10980add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 --2- 172.21.15.106:0/3028761000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10980cd650 0x7f10980add80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f109002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 -- 172.21.15.106:0/3028761000 shutdown_connections 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 --2- 172.21.15.106:0/3028761000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10980cd650 0x7f10980add80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 -- 172.21.15.106:0/3028761000 >> 172.21.15.106:0/3028761000 conn(0x7f10980b2d50 msgr2=0x7f10980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 -- 172.21.15.106:0/3028761000 shutdown_connections 2024-04-26T21:46:55.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.792+0000 7f109da00640 1 -- 172.21.15.106:0/3028761000 wait complete. 2024-04-26T21:46:55.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.794+0000 7f109da00640 1 Processor -- start 2024-04-26T21:46:55.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.794+0000 7f109da00640 1 -- start start 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f109806ce80 0x7f1098152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10980cd650 0x7f10981f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f109815d1d0 con 0x7f109815cdf0 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10980b05b0 con 0x7f10980cd650 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f109da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f109815d350 con 0x7f109806ce80 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f1097400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f1097400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53416/0 (socket says 172.21.15.106:53416) 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.795+0000 7f1097400640 1 -- 172.21.15.106:0/1981421318 learned_addr learned my addr 172.21.15.106:0/1981421318 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1096000640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10980cd650 0x7f10981f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1096a00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f109806ce80 0x7f1098152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 -- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f109806ce80 msgr2=0x7f1098152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f109806ce80 0x7f1098152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 -- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10980cd650 msgr2=0x7f10981f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10980cd650 0x7f10981f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 -- 172.21.15.106:0/1981421318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981603f0 con 0x7f109815cdf0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.796+0000 7f1097400640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f1090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.797+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090040070 con 0x7f109815cdf0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.797+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1090031c90 con 0x7f109815cdf0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.797+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981f93b0 con 0x7f109815cdf0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.797+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109003c640 con 0x7f109815cdf0 2024-04-26T21:46:55.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.797+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109809c720 con 0x7f109815cdf0 2024-04-26T21:46:55.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.798+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109809c960 con 0x7f109815cdf0 2024-04-26T21:46:55.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.799+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1090039040 con 0x7f109815cdf0 2024-04-26T21:46:55.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.800+0000 7f108ea00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.800+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f10900ad250 con 0x7f109815cdf0 2024-04-26T21:46:55.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.800+0000 7f1096a00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.801+0000 7f1096a00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 0x7f107806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1088006fd0 tx=0x7f1088008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:55.804+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f109809c960 con 0x7f109815cdf0 2024-04-26T21:46:56.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.017+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1098069540 con 0x7f109815cdf0 2024-04-26T21:46:56.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.018+0000 7f108ea00640 1 -- 172.21.15.106:0/1981421318 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f109007b6b0 con 0x7f109815cdf0 2024-04-26T21:46:56.019 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:56.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 msgr2=0x7f107806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 0x7f107806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1088006fd0 tx=0x7f1088008040 comp rx=0 tx=0).stop 2024-04-26T21:46:56.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 msgr2=0x7f109809ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f1090004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 shutdown_connections 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f109815cdf0 0x7f109809ba50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10980cd650 0x7f10981f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 --2- 172.21.15.106:0/1981421318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f109806ce80 0x7f1098152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 >> 172.21.15.106:0/1981421318 conn(0x7f10980b2d50 msgr2=0x7f109806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 shutdown_connections 2024-04-26T21:46:56.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.021+0000 7f109da00640 1 -- 172.21.15.106:0/1981421318 wait complete. 2024-04-26T21:46:56.041 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:46:56.041 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:56.097 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.297+0000 7f32e5000640 1 Processor -- start 2024-04-26T21:46:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.297+0000 7f32e5000640 1 -- start start 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32e5000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32e006ce80 0x7f32e00add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32e5000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32e00cd650 0x7f32e00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32e5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e00af9a0 con 0x7f32e015cdf0 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32e5000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f32e00ae2c0 con 0x7f32e00cd650 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32e5000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f32e00b05b0 con 0x7f32e006ce80 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.299+0000 7f32de800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32e00cd650 0x7f32e00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32e00cd650 0x7f32e00b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53420/0 (socket says 172.21.15.106:53420) 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 -- 172.21.15.106:0/3235410260 learned_addr learned my addr 172.21.15.106:0/3235410260 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32d7e00640 1 --2- 172.21.15.106:0/3235410260 >> v2:172.21.15.204:3300/0 conn(0x7f32e006ce80 0x7f32e00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 -- 172.21.15.106:0/3235410260 >> v2:172.21.15.204:3300/0 conn(0x7f32e006ce80 msgr2=0x7f32e00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 --2- 172.21.15.106:0/3235410260 >> v2:172.21.15.204:3300/0 conn(0x7f32e006ce80 0x7f32e00add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 -- 172.21.15.106:0/3235410260 >> v1:172.21.15.204:6789/0 conn(0x7f32e015cdf0 legacy=0x7f32e006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.300+0000 7f32de800640 1 -- 172.21.15.106:0/3235410260 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00afc80 con 0x7f32e00cd650 2024-04-26T21:46:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.301+0000 7f32de800640 1 --2- 172.21.15.106:0/3235410260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00b3d10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f32e00d1d00 tx=0x7f32d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdcd08c548ff64a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.301+0000 7f32dd400640 1 -- 172.21.15.106:0/3235410260 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8036070 con 0x7f32e00cd650 2024-04-26T21:46:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.301+0000 7f32dd400640 1 -- 172.21.15.106:0/3235410260 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d8031600 con 0x7f32e00cd650 2024-04-26T21:46:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.301+0000 7f32dd400640 1 -- 172.21.15.106:0/3235410260 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80319d0 con 0x7f32e00cd650 2024-04-26T21:46:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 -- 172.21.15.106:0/3235410260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 msgr2=0x7f32e00b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 --2- 172.21.15.106:0/3235410260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00b3d10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f32e00d1d00 tx=0x7f32d802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 -- 172.21.15.106:0/3235410260 shutdown_connections 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 --2- 172.21.15.106:0/3235410260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00b3d10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 --2- 172.21.15.106:0/3235410260 >> v2:172.21.15.204:3300/0 conn(0x7f32e006ce80 0x7f32e00add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.302+0000 7f32e5000640 1 -- 172.21.15.106:0/3235410260 >> 172.21.15.106:0/3235410260 conn(0x7f32e00b2d50 msgr2=0x7f32e00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.303+0000 7f32e5000640 1 -- 172.21.15.106:0/3235410260 shutdown_connections 2024-04-26T21:46:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.303+0000 7f32e5000640 1 -- 172.21.15.106:0/3235410260 wait complete. 2024-04-26T21:46:56.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.304+0000 7f32e5000640 1 Processor -- start 2024-04-26T21:46:56.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.304+0000 7f32e5000640 1 -- start start 2024-04-26T21:46:56.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.305+0000 7f32e5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 0x7f32e0154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.305+0000 7f32e5000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32e5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 0x7f32e00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32e5000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32e015d1d0 con 0x7f32e00cd650 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32e5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00b0240 con 0x7f32e006ce80 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32e5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f32e015d350 con 0x7f32e015cdf0 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53428/0 (socket says 172.21.15.106:53428) 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 -- 172.21.15.106:0/4070713419 learned_addr learned my addr 172.21.15.106:0/4070713419 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32dde00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 0x7f32e0154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32de800640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 0x7f32e00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 -- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 msgr2=0x7f32e00b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 0x7f32e00b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 -- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 msgr2=0x7f32e0154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 0x7f32e0154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32d7e00640 1 -- 172.21.15.106:0/4070713419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00d0030 con 0x7f32e00cd650 2024-04-26T21:46:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32de800640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 0x7f32e00b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.306+0000 7f32dde00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 0x7f32e0154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32d7e00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d800 tx=0x7f32c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800b6c0 con 0x7f32e00cd650 2024-04-26T21:46:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c8019070 con 0x7f32e00cd650 2024-04-26T21:46:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800ba80 con 0x7f32e00cd650 2024-04-26T21:46:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e009de00 con 0x7f32e00cd650 2024-04-26T21:46:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.307+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e009e3c0 con 0x7f32e00cd650 2024-04-26T21:46:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.309+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e0069540 con 0x7f32e00cd650 2024-04-26T21:46:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.309+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32c800bc20 con 0x7f32e00cd650 2024-04-26T21:46:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.310+0000 7f32d6000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 0x7f32bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.310+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f32c808c160 con 0x7f32e00cd650 2024-04-26T21:46:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.311+0000 7f32dde00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 0x7f32bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.311+0000 7f32dde00640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 0x7f32bc06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32e0154a70 tx=0x7f32d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.315+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32c805a470 con 0x7f32e00cd650 2024-04-26T21:46:56.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.512+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32e00b5230 con 0x7f32e00cd650 2024-04-26T21:46:56.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.515+0000 7f32d6000640 1 -- 172.21.15.106:0/4070713419 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f32c8059d70 con 0x7f32e00cd650 2024-04-26T21:46:56.516 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:56.517 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 msgr2=0x7f32bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 0x7f32bc06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32e0154a70 tx=0x7f32d0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 msgr2=0x7f32e00cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d800 tx=0x7f32c800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 shutdown_connections 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f32bc06c8f0 0x7f32bc06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32e015cdf0 0x7f32e00b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32e00cd650 0x7f32e00cf9a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 --2- 172.21.15.106:0/4070713419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32e006ce80 0x7f32e0154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 >> 172.21.15.106:0/4070713419 conn(0x7f32e00b2d50 msgr2=0x7f32e006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 shutdown_connections 2024-04-26T21:46:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.518+0000 7f32e5000640 1 -- 172.21.15.106:0/4070713419 wait complete. 2024-04-26T21:46:56.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:56.537 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:56.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.731+0000 7fca1f200640 1 Processor -- start 2024-04-26T21:46:56.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.731+0000 7fca1f200640 1 -- start start 2024-04-26T21:46:56.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.732+0000 7fca1f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20161140 0x7fca2006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.732+0000 7fca1f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fca20146280 0x7fca201466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.732+0000 7fca1f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca200b4d20 con 0x7fca20160d60 2024-04-26T21:46:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.732+0000 7fca1f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fca20146c30 con 0x7fca20146280 2024-04-26T21:46:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.732+0000 7fca1f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca200b5000 con 0x7fca20161140 2024-04-26T21:46:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.733+0000 7fca1e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fca20146280 0x7fca201466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.733+0000 7fca1d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20161140 0x7fca2006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.733+0000 7fca1e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fca20146280 0x7fca201466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:47662/0 (socket says 172.21.15.106:47662) 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.733+0000 7fca1e800640 1 -- 172.21.15.106:0/1087099780 learned_addr learned my addr 172.21.15.106:0/1087099780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.734+0000 7fca1e800640 1 -- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20161140 msgr2=0x7fca2006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.734+0000 7fca1e800640 1 --2- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20161140 0x7fca2006dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.734+0000 7fca1e800640 1 -- 172.21.15.106:0/1087099780 >> v1:172.21.15.204:6789/0 conn(0x7fca20160d60 legacy=0x7fca200b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.734+0000 7fca1e800640 1 -- 172.21.15.106:0/1087099780 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca200b52e0 con 0x7fca20146280 2024-04-26T21:46:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.736+0000 7fca1e800640 1 --2- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca201466f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca18009870 tx=0x7fca1802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b353d3a9bb8df247 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.736+0000 7fca1ca00640 1 -- 172.21.15.106:0/1087099780 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18037070 con 0x7fca20146280 2024-04-26T21:46:56.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.736+0000 7fca1ca00640 1 -- 172.21.15.106:0/1087099780 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca18032440 con 0x7fca20146280 2024-04-26T21:46:56.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.736+0000 7fca1ca00640 1 -- 172.21.15.106:0/1087099780 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18032810 con 0x7fca20146280 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 -- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 msgr2=0x7fca201466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 --2- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca201466f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca18009870 tx=0x7fca1802a270 comp rx=0 tx=0).stop 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 -- 172.21.15.106:0/1087099780 shutdown_connections 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 --2- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca201466f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 --2- 172.21.15.106:0/1087099780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20161140 0x7fca2006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 -- 172.21.15.106:0/1087099780 >> 172.21.15.106:0/1087099780 conn(0x7fca200b2900 msgr2=0x7fca200b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 -- 172.21.15.106:0/1087099780 shutdown_connections 2024-04-26T21:46:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.737+0000 7fca1f200640 1 -- 172.21.15.106:0/1087099780 wait complete. 2024-04-26T21:46:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.739+0000 7fca1f200640 1 Processor -- start 2024-04-26T21:46:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.739+0000 7fca1f200640 1 -- start start 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca2013d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20160d60 0x7fca2015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca2006e420 con 0x7fca20161140 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca200cf6f0 con 0x7fca20146280 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca2006e5a0 con 0x7fca20160d60 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53444/0 (socket says 172.21.15.106:53444) 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1e800640 1 -- 172.21.15.106:0/1652369311 learned_addr learned my addr 172.21.15.106:0/1652369311 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.740+0000 7fca1de00640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca2013d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 -- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20160d60 msgr2=0x7fca2015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20160d60 0x7fca2015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 -- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 msgr2=0x7fca2013d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca2013d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 -- 172.21.15.106:0/1652369311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2013ce10 con 0x7fca20161140 2024-04-26T21:46:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1de00640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca2013d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1e800640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fca1802a2b0 tx=0x7fca180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18041020 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1803d620 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca20140360 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18037040 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.741+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2013fe50 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.743+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca201400e0 con 0x7fca20161140 2024-04-26T21:46:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.744+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca18044020 con 0x7fca20161140 2024-04-26T21:46:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.745+0000 7fca16000640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 0x7fc9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.745+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fca180abf60 con 0x7fca20161140 2024-04-26T21:46:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.745+0000 7fca1de00640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 0x7fc9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.746+0000 7fca1de00640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 0x7fc9fc06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca10001d70 tx=0x7fca10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.750+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca201400e0 con 0x7fca20161140 2024-04-26T21:46:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.945+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca200693c0 con 0x7fca20161140 2024-04-26T21:46:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.945+0000 7fca16000640 1 -- 172.21.15.106:0/1652369311 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fca1804b070 con 0x7fca20161140 2024-04-26T21:46:56.947 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 msgr2=0x7fc9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 0x7fc9fc06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca10001d70 tx=0x7fca10009040 comp rx=0 tx=0).stop 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 msgr2=0x7fca2013f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fca1802a2b0 tx=0x7fca180046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 shutdown_connections 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca20161140 0x7fca2013f910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc9fc06c8f0 0x7fc9fc06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca20160d60 0x7fca2015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 --2- 172.21.15.106:0/1652369311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca20146280 0x7fca2013d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 >> 172.21.15.106:0/1652369311 conn(0x7fca200b2900 msgr2=0x7fca200d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 shutdown_connections 2024-04-26T21:46:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:56.948+0000 7fca1f200640 1 -- 172.21.15.106:0/1652369311 wait complete. 2024-04-26T21:46:56.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:57.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.161+0000 7f70bf200640 1 Processor -- start 2024-04-26T21:46:57.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.161+0000 7f70bf200640 1 -- start start 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bf200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c0071e80 0x7f70c00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 0x7f70c00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c006e320 con 0x7f70c00b4ed0 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bf200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f70c006e600 con 0x7f70c0071e80 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f70c006e8e0 con 0x7f70c0072970 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bde00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c0071e80 0x7f70c00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bde00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c0071e80 0x7f70c00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53458/0 (socket says 172.21.15.106:53458) 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.163+0000 7f70bde00640 1 -- 172.21.15.106:0/3904332317 learned_addr learned my addr 172.21.15.106:0/3904332317 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70be800640 1 --2- 172.21.15.106:0/3904332317 >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 0x7f70c00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70bde00640 1 -- 172.21.15.106:0/3904332317 >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 msgr2=0x7f70c00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70bde00640 1 --2- 172.21.15.106:0/3904332317 >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 0x7f70c00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70bde00640 1 -- 172.21.15.106:0/3904332317 >> v1:172.21.15.204:6789/0 conn(0x7f70c00b4ed0 legacy=0x7f70c0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70bde00640 1 -- 172.21.15.106:0/3904332317 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c006ebc0 con 0x7f70c0071e80 2024-04-26T21:46:57.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.164+0000 7f70be800640 1 --2- 172.21.15.106:0/3904332317 >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 0x7f70c00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.165+0000 7f70bde00640 1 --2- 172.21.15.106:0/3904332317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0071e80 0x7f70c00b3ee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c0073d20 tx=0x7f70a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4f28cf7cc75a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.165+0000 7f70bca00640 1 -- 172.21.15.106:0/3904332317 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8037070 con 0x7f70c0071e80 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.165+0000 7f70bca00640 1 -- 172.21.15.106:0/3904332317 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a8032440 con 0x7f70c0071e80 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.165+0000 7f70bca00640 1 -- 172.21.15.106:0/3904332317 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8032810 con 0x7f70c0071e80 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 -- 172.21.15.106:0/3904332317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0071e80 msgr2=0x7f70c00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 --2- 172.21.15.106:0/3904332317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0071e80 0x7f70c00b3ee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c0073d20 tx=0x7f70a802a230 comp rx=0 tx=0).stop 2024-04-26T21:46:57.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 -- 172.21.15.106:0/3904332317 shutdown_connections 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 --2- 172.21.15.106:0/3904332317 >> v2:172.21.15.204:3300/0 conn(0x7f70c0072970 0x7f70c00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 --2- 172.21.15.106:0/3904332317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0071e80 0x7f70c00b3ee0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 -- 172.21.15.106:0/3904332317 >> 172.21.15.106:0/3904332317 conn(0x7f70c00b2900 msgr2=0x7f70c00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 -- 172.21.15.106:0/3904332317 shutdown_connections 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.166+0000 7f70bf200640 1 -- 172.21.15.106:0/3904332317 wait complete. 2024-04-26T21:46:57.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.168+0000 7f70bf200640 1 Processor -- start 2024-04-26T21:46:57.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.168+0000 7f70bf200640 1 -- start start 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70c0071e80 0x7f70c00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70c00b4ed0 0x7f70c013e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70c006f0e0 con 0x7f70c0072970 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00b5ff0 con 0x7f70c0071e80 2024-04-26T21:46:57.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f70c006f260 con 0x7f70c00b4ed0 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53464/0 (socket says 172.21.15.106:53464) 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.169+0000 7f70bd400640 1 -- 172.21.15.106:0/2788342227 learned_addr learned my addr 172.21.15.106:0/2788342227 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bde00640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70c0071e80 0x7f70c00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 -- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70c00b4ed0 msgr2=0x7f70c013e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70c00b4ed0 0x7f70c013e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 -- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70c0071e80 msgr2=0x7f70c00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70c0071e80 0x7f70c00cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 -- 172.21.15.106:0/2788342227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c0153910 con 0x7f70c0072970 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bd400640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b3e0 tx=0x7f70b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0015020 con 0x7f70c0072970 2024-04-26T21:46:57.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b0011850 con 0x7f70c0072970 2024-04-26T21:46:57.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0011c40 con 0x7f70c0072970 2024-04-26T21:46:57.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c0153bf0 con 0x7f70c0072970 2024-04-26T21:46:57.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.170+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01541b0 con 0x7f70c0072970 2024-04-26T21:46:57.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.172+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c00693c0 con 0x7f70c0072970 2024-04-26T21:46:57.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.172+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70b0018020 con 0x7f70c0072970 2024-04-26T21:46:57.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.174+0000 7f70b6000640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 0x7f709c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.174+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f70b008bc30 con 0x7f70c0072970 2024-04-26T21:46:57.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.174+0000 7f70bde00640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 0x7f709c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.175+0000 7f70bde00640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 0x7f709c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f70a8034000 tx=0x7f70a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.178+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70b001f020 con 0x7f70c0072970 2024-04-26T21:46:57.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.377+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70c00ccfc0 con 0x7f70c0072970 2024-04-26T21:46:57.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.378+0000 7f70b6000640 1 -- 172.21.15.106:0/2788342227 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f70b0059f40 con 0x7f70c0072970 2024-04-26T21:46:57.379 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.380+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 msgr2=0x7f709c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.380+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 0x7f709c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f70a8034000 tx=0x7f70a802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.380+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 msgr2=0x7f70c00cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.380+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b3e0 tx=0x7f70b000b8a0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 shutdown_connections 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f709c06c8f0 0x7f709c06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70c00b4ed0 0x7f70c013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c0072970 0x7f70c00cf740 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 --2- 172.21.15.106:0/2788342227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70c0071e80 0x7f70c00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 >> 172.21.15.106:0/2788342227 conn(0x7f70c00b2900 msgr2=0x7f70c0072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 shutdown_connections 2024-04-26T21:46:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.381+0000 7f70bf200640 1 -- 172.21.15.106:0/2788342227 wait complete. 2024-04-26T21:46:57.401 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:57.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.596+0000 7f6bea000640 1 Processor -- start 2024-04-26T21:46:57.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.597+0000 7f6bea000640 1 -- start start 2024-04-26T21:46:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.598+0000 7f6bea000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6be40b6210 0x7f6be416a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.598+0000 7f6bea000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be40b6710 con 0x7f6be40d3110 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.598+0000 7f6bea000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be40d3600 con 0x7f6be41632e0 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.598+0000 7f6bea000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6be40d38e0 con 0x7f6be40b6210 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be9600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6be40b6210 0x7f6be416a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be3e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6be40d3110 0x7f6be4162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48230/0 (socket says 172.21.15.106:48230) 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be3e00640 1 -- 172.21.15.106:0/3639927659 learned_addr learned my addr 172.21.15.106:0/3639927659 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be9600640 1 -- 172.21.15.106:0/3639927659 >> v1:172.21.15.204:6789/0 conn(0x7f6be41632e0 legacy=0x7f6be41636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be9600640 1 -- 172.21.15.106:0/3639927659 >> v1:172.21.15.106:6789/0 conn(0x7f6be40d3110 legacy=0x7f6be4162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.599+0000 7f6be9600640 1 -- 172.21.15.106:0/3639927659 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40d3bc0 con 0x7f6be40b6210 2024-04-26T21:46:57.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.600+0000 7f6be9600640 1 --2- 172.21.15.106:0/3639927659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be40b6210 0x7f6be416a3a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0097a0 tx=0x7f6bdc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4437279971364b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.600+0000 7f6be3400640 1 -- 172.21.15.106:0/3639927659 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc037070 con 0x7f6be40b6210 2024-04-26T21:46:57.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.600+0000 7f6be3400640 1 -- 172.21.15.106:0/3639927659 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bdc02ae50 con 0x7f6be40b6210 2024-04-26T21:46:57.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.600+0000 7f6be3400640 1 -- 172.21.15.106:0/3639927659 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc032660 con 0x7f6be40b6210 2024-04-26T21:46:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 -- 172.21.15.106:0/3639927659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be40b6210 msgr2=0x7f6be416a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 --2- 172.21.15.106:0/3639927659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be40b6210 0x7f6be416a3a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0097a0 tx=0x7f6bdc02a140 comp rx=0 tx=0).stop 2024-04-26T21:46:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 -- 172.21.15.106:0/3639927659 shutdown_connections 2024-04-26T21:46:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 --2- 172.21.15.106:0/3639927659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be40b6210 0x7f6be416a3a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 -- 172.21.15.106:0/3639927659 >> 172.21.15.106:0/3639927659 conn(0x7f6be409e2f0 msgr2=0x7f6be40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 -- 172.21.15.106:0/3639927659 shutdown_connections 2024-04-26T21:46:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.601+0000 7f6bea000640 1 -- 172.21.15.106:0/3639927659 wait complete. 2024-04-26T21:46:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.603+0000 7f6bea000640 1 Processor -- start 2024-04-26T21:46:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.603+0000 7f6bea000640 1 -- start start 2024-04-26T21:46:57.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 0x7f6be40b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 0x7f6be415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40b3260 con 0x7f6be41632e0 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4163dd0 con 0x7f6be40b6210 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6bea000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6be40b33e0 con 0x7f6be40d3110 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6be9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6be9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53486/0 (socket says 172.21.15.106:53486) 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.604+0000 7f6be9600640 1 -- 172.21.15.106:0/1342207517 learned_addr learned my addr 172.21.15.106:0/1342207517 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be8c00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 0x7f6be40b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be3e00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 0x7f6be415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 -- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 msgr2=0x7f6be415b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 0x7f6be415b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 -- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 msgr2=0x7f6be40b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 0x7f6be40b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 -- 172.21.15.106:0/1342207517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40b87c0 con 0x7f6be41632e0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be8c00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 0x7f6be40b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be3e00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 0x7f6be415b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be9600640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098c0 tx=0x7f6bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc042070 con 0x7f6be41632e0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bdc032e10 con 0x7f6be41632e0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc037040 con 0x7f6be41632e0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.605+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be40b8aa0 con 0x7f6be41632e0 2024-04-26T21:46:57.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.606+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be40b9060 con 0x7f6be41632e0 2024-04-26T21:46:57.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.607+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be40693c0 con 0x7f6be41632e0 2024-04-26T21:46:57.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.608+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bdc045070 con 0x7f6be41632e0 2024-04-26T21:46:57.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.608+0000 7f6be0c00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 0x7f6bc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.609+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6bdc0ac210 con 0x7f6be41632e0 2024-04-26T21:46:57.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.609+0000 7f6be8c00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 0x7f6bc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.609+0000 7f6be8c00640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 0x7f6bc406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6be40b55e0 tx=0x7f6bd40073c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.613+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6bdc07a5b0 con 0x7f6be41632e0 2024-04-26T21:46:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.803+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6be406e420 con 0x7f6be41632e0 2024-04-26T21:46:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.804+0000 7f6be0c00640 1 -- 172.21.15.106:0/1342207517 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f6bdc079eb0 con 0x7f6be41632e0 2024-04-26T21:46:57.805 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 msgr2=0x7f6bc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 0x7f6bc406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6be40b55e0 tx=0x7f6bd40073c0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 msgr2=0x7f6be415be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098c0 tx=0x7f6bdc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 shutdown_connections 2024-04-26T21:46:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6be41632e0 0x7f6be415be30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6bc406c8f0 0x7f6bc406edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6be40d3110 0x7f6be415b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 --2- 172.21.15.106:0/1342207517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6be40b6210 0x7f6be40b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 >> 172.21.15.106:0/1342207517 conn(0x7f6be409e2f0 msgr2=0x7f6be40b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 shutdown_connections 2024-04-26T21:46:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:57.806+0000 7f6bea000640 1 -- 172.21.15.106:0/1342207517 wait complete. 2024-04-26T21:46:57.825 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.012+0000 7f28ec200640 1 Processor -- start 2024-04-26T21:46:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.013+0000 7f28ec200640 1 -- start start 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ec200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28e406ce80 0x7f28e40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ec200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e40af9a0 con 0x7f28e40cd650 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ec200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e40afc80 con 0x7f28e415cdf0 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ec200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f28e40ae2c0 con 0x7f28e406ce80 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28e8e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28e406ce80 0x7f28e40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ea200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f28e40cd650 0x7f28e40b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40284/0 (socket says 172.21.15.106:40284) 2024-04-26T21:46:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.014+0000 7f28ea200640 1 -- 172.21.15.106:0/942795821 learned_addr learned my addr 172.21.15.106:0/942795821 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.015+0000 7f28e8e00640 1 -- 172.21.15.106:0/942795821 >> v1:172.21.15.204:6789/0 conn(0x7f28e415cdf0 legacy=0x7f28e406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.015+0000 7f28e8e00640 1 -- 172.21.15.106:0/942795821 >> v1:172.21.15.106:6789/0 conn(0x7f28e40cd650 legacy=0x7f28e40b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.015+0000 7f28e8e00640 1 -- 172.21.15.106:0/942795821 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e40aff60 con 0x7f28e406ce80 2024-04-26T21:46:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.015+0000 7f28e3e00640 1 -- 172.21.15.106:0/942795821 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794951488 0 0) 0x7f28e40af9a0 con 0x7f28e40cd650 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.016+0000 7f28e8e00640 1 --2- 172.21.15.106:0/942795821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e406ce80 0x7f28e40add80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f28d00097a0 tx=0x7f28d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44df292b9a947651 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.016+0000 7f28e3e00640 1 -- 172.21.15.106:0/942795821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037070 con 0x7f28e406ce80 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.016+0000 7f28e3e00640 1 -- 172.21.15.106:0/942795821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0032440 con 0x7f28e406ce80 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.016+0000 7f28e3e00640 1 -- 172.21.15.106:0/942795821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0032810 con 0x7f28e406ce80 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 -- 172.21.15.106:0/942795821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e406ce80 msgr2=0x7f28e40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 --2- 172.21.15.106:0/942795821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e406ce80 0x7f28e40add80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f28d00097a0 tx=0x7f28d002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 -- 172.21.15.106:0/942795821 shutdown_connections 2024-04-26T21:46:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 --2- 172.21.15.106:0/942795821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e406ce80 0x7f28e40add80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 -- 172.21.15.106:0/942795821 >> 172.21.15.106:0/942795821 conn(0x7f28e40b2d50 msgr2=0x7f28e40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 -- 172.21.15.106:0/942795821 shutdown_connections 2024-04-26T21:46:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.017+0000 7f28ec200640 1 -- 172.21.15.106:0/942795821 wait complete. 2024-04-26T21:46:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.019+0000 7f28ec200640 1 Processor -- start 2024-04-26T21:46:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.019+0000 7f28ec200640 1 -- start start 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28e406ce80 0x7f28e40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28e415cdf0 0x7f28e414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28e415d1d0 con 0x7f28e40cd650 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28e40b05b0 con 0x7f28e415cdf0 2024-04-26T21:46:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28ec200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28e415d350 con 0x7f28e406ce80 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59614/0 (socket says 172.21.15.106:59614) 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 -- 172.21.15.106:0/1721419926 learned_addr learned my addr 172.21.15.106:0/1721419926 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 -- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28e406ce80 msgr2=0x7f28e40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28e406ce80 0x7f28e40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 -- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28e415cdf0 msgr2=0x7f28e414e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28e415cdf0 0x7f28e414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.020+0000 7f28e8e00640 1 -- 172.21.15.106:0/1721419926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e415ff00 con 0x7f28e40cd650 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.021+0000 7f28e8e00640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f28d0035040 tx=0x7f28d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.021+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037040 con 0x7f28e40cd650 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.021+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0004250 con 0x7f28e40cd650 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.021+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0032e20 con 0x7f28e40cd650 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.021+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41601e0 con 0x7f28e40cd650 2024-04-26T21:46:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.022+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e415bed0 con 0x7f28e40cd650 2024-04-26T21:46:58.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.023+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e4069540 con 0x7f28e40cd650 2024-04-26T21:46:58.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.024+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28d0030770 con 0x7f28e40cd650 2024-04-26T21:46:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.024+0000 7f28e1600640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.025+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f28d00acf90 con 0x7f28e40cd650 2024-04-26T21:46:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.025+0000 7f28e9800640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.025+0000 7f28e9800640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 0x7f28c406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.029+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28d007b3c0 con 0x7f28e40cd650 2024-04-26T21:46:58.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.242+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28e41603b0 con 0x7f28e40cd650 2024-04-26T21:46:58.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.243+0000 7f28e1600640 1 -- 172.21.15.106:0/1721419926 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f28d007acc0 con 0x7f28e40cd650 2024-04-26T21:46:58.244 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 msgr2=0x7f28c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 0x7f28c406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4006bc0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 msgr2=0x7f28e415b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f28d0035040 tx=0x7f28d00046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 shutdown_connections 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28e415cdf0 0x7f28e414e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28e40cd650 0x7f28e415b990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 --2- 172.21.15.106:0/1721419926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28e406ce80 0x7f28e40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.245+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 >> 172.21.15.106:0/1721419926 conn(0x7f28e40b2d50 msgr2=0x7f28e40b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.246+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 shutdown_connections 2024-04-26T21:46:58.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.246+0000 7f28ec200640 1 -- 172.21.15.106:0/1721419926 wait complete. 2024-04-26T21:46:58.265 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-04-26T21:46:58.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:58.266 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-26T21:46:58.354 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:46:58.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:46:58.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:58.355 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T21:46:58.399 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:46:58.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:58.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.605+0000 7f620f400640 1 Processor -- start 2024-04-26T21:46:58.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.605+0000 7f620f400640 1 -- start start 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.606+0000 7f620f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f620815cb70 0x7f620806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.606+0000 7f620f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f620815cfd0 0x7f620806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.606+0000 7f620f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62080ae850 con 0x7f620806c770 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.606+0000 7f620f400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f62080af460 con 0x7f620815cb70 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.606+0000 7f620f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f62080aeb30 con 0x7f620815cfd0 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f620815cb70 0x7f620806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f620815cb70 0x7f620806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59624/0 (socket says 172.21.15.106:59624) 2024-04-26T21:46:58.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 -- 172.21.15.106:0/2386985319 learned_addr learned my addr 172.21.15.106:0/2386985319 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:58.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f620d400640 1 --2- 172.21.15.106:0/2386985319 >> v2:172.21.15.204:3300/0 conn(0x7f620815cfd0 0x7f620806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 -- 172.21.15.106:0/2386985319 >> v2:172.21.15.204:3300/0 conn(0x7f620815cfd0 msgr2=0x7f620806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:58.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 --2- 172.21.15.106:0/2386985319 >> v2:172.21.15.204:3300/0 conn(0x7f620815cfd0 0x7f620806f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 -- 172.21.15.106:0/2386985319 >> v1:172.21.15.204:6789/0 conn(0x7f620806c770 legacy=0x7f620806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:58.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.607+0000 7f6207e00640 1 -- 172.21.15.106:0/2386985319 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62080aee10 con 0x7f620815cb70 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.608+0000 7f6207e00640 1 --2- 172.21.15.106:0/2386985319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f620806ee60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62080b0870 tx=0x7f61f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844c42fbc4546272 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.608+0000 7f6207400640 1 -- 172.21.15.106:0/2386985319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4037070 con 0x7f620815cb70 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f6207400640 1 -- 172.21.15.106:0/2386985319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f4032440 con 0x7f620815cb70 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f6207400640 1 -- 172.21.15.106:0/2386985319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4032810 con 0x7f620815cb70 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 -- 172.21.15.106:0/2386985319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 msgr2=0x7f620806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 --2- 172.21.15.106:0/2386985319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f620806ee60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62080b0870 tx=0x7f61f402a230 comp rx=0 tx=0).stop 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 -- 172.21.15.106:0/2386985319 shutdown_connections 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 --2- 172.21.15.106:0/2386985319 >> v2:172.21.15.204:3300/0 conn(0x7f620815cfd0 0x7f620806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 --2- 172.21.15.106:0/2386985319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f620806ee60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 -- 172.21.15.106:0/2386985319 >> 172.21.15.106:0/2386985319 conn(0x7f62080b2d50 msgr2=0x7f62080b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 -- 172.21.15.106:0/2386985319 shutdown_connections 2024-04-26T21:46:58.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.609+0000 7f620f400640 1 -- 172.21.15.106:0/2386985319 wait complete. 2024-04-26T21:46:58.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.611+0000 7f620f400640 1 Processor -- start 2024-04-26T21:46:58.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.612+0000 7f620f400640 1 -- start start 2024-04-26T21:46:58.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f620806c770 0x7f6208154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f620815cfd0 0x7f620814fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f620815c780 con 0x7f620815cb70 2024-04-26T21:46:58.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f620815c600 con 0x7f620815cfd0 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62080af8f0 con 0x7f620806c770 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f6207e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f6207e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59638/0 (socket says 172.21.15.106:59638) 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f6207e00640 1 -- 172.21.15.106:0/3706880249 learned_addr learned my addr 172.21.15.106:0/3706880249 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f6207e00640 1 -- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f620806c770 msgr2=0x7f6208154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620d400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f620815cfd0 0x7f620814fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.613+0000 7f620ca00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f620806c770 0x7f6208154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6207e00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f620806c770 0x7f6208154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6207e00640 1 -- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f620815cfd0 msgr2=0x7f620814fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6207e00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f620815cfd0 0x7f620814fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6207e00640 1 -- 172.21.15.106:0/3706880249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6208151320 con 0x7f620815cb70 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6207e00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4041020 con 0x7f620815cb70 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f4030970 con 0x7f620815cb70 2024-04-26T21:46:58.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4037040 con 0x7f620815cb70 2024-04-26T21:46:58.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081515c0 con 0x7f620815cb70 2024-04-26T21:46:58.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.614+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6208151b80 con 0x7f620815cb70 2024-04-26T21:46:58.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.616+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62080ca0f0 con 0x7f620815cb70 2024-04-26T21:46:58.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.616+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61f4044020 con 0x7f620815cb70 2024-04-26T21:46:58.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.617+0000 7f6204c00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 0x7f61e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.617+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f61f40ac380 con 0x7f620815cb70 2024-04-26T21:46:58.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.618+0000 7f620ca00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 0x7f61e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.618+0000 7f620ca00640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 0x7f61e806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00a730 tx=0x7f61fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.622+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62080ca0f0 con 0x7f620815cb70 2024-04-26T21:46:58.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.862+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62080cd430 con 0x7f620815cb70 2024-04-26T21:46:58.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.863+0000 7f6204c00640 1 -- 172.21.15.106:0/3706880249 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f61f404b070 con 0x7f620815cb70 2024-04-26T21:46:58.864 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:58.865 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:46:58.865 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:46:58.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 msgr2=0x7f61e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 0x7f61e806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00a730 tx=0x7f61fc008040 comp rx=0 tx=0).stop 2024-04-26T21:46:58.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 msgr2=0x7f6208150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f40046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 shutdown_connections 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f620815cfd0 0x7f620814fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f620815cb70 0x7f6208150de0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f61e806c8f0 0x7f61e806edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 --2- 172.21.15.106:0/3706880249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f620806c770 0x7f6208154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 >> 172.21.15.106:0/3706880249 conn(0x7f62080b2d50 msgr2=0x7f62080b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 shutdown_connections 2024-04-26T21:46:58.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:58.866+0000 7f620f400640 1 -- 172.21.15.106:0/3706880249 wait complete. 2024-04-26T21:46:58.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:59.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.069+0000 7f1fa8800640 1 Processor -- start 2024-04-26T21:46:59.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.069+0000 7f1fa8800640 1 -- start start 2024-04-26T21:46:59.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.070+0000 7f1fa8800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1fa00d5d90 0x7f1fa015f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.070+0000 7f1fa8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00d3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.070+0000 7f1fa8800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa00b5d30 con 0x7f1fa015ef60 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.070+0000 7f1fa8800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa00b6010 con 0x7f1fa00d5d90 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.070+0000 7f1fa8800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa00b62f0 con 0x7f1fa00d34a0 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1fa015ef60 0x7f1fa00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40296/0 (socket says 172.21.15.106:40296) 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5e00640 1 -- 172.21.15.106:0/2250290191 learned_addr learned my addr 172.21.15.106:0/2250290191 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:59.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5400640 1 --2- 172.21.15.106:0/2250290191 >> v2:172.21.15.106:3300/0 conn(0x7f1fa00d5d90 0x7f1fa015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa4a00640 1 -- 172.21.15.106:0/2250290191 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145015643 0 0) 0x7f1fa00b5d30 con 0x7f1fa015ef60 2024-04-26T21:46:59.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa4a00640 1 -- 172.21.15.106:0/2250290191 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f88003520 con 0x7f1fa015ef60 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa6800640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5400640 1 -- 172.21.15.106:0/2250290191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 msgr2=0x7f1fa00d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5400640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00d3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5400640 1 -- 172.21.15.106:0/2250290191 >> v1:172.21.15.106:6789/0 conn(0x7f1fa015ef60 legacy=0x7f1fa00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.071+0000 7f1fa5400640 1 -- 172.21.15.106:0/2250290191 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa00b65d0 con 0x7f1fa00d5d90 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.072+0000 7f1fa5400640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa015f340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f900097a0 tx=0x7f1f9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11630116302ce5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa4a00640 1 -- 172.21.15.106:0/2250290191 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90037070 con 0x7f1fa00d5d90 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa4a00640 1 -- 172.21.15.106:0/2250290191 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90032440 con 0x7f1fa00d5d90 2024-04-26T21:46:59.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa4a00640 1 -- 172.21.15.106:0/2250290191 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90032810 con 0x7f1fa00d5d90 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 -- 172.21.15.106:0/2250290191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 msgr2=0x7f1fa015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa015f340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f900097a0 tx=0x7f1f9002a210 comp rx=0 tx=0).stop 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 -- 172.21.15.106:0/2250290191 shutdown_connections 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 --2- 172.21.15.106:0/2250290191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa015f340 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 -- 172.21.15.106:0/2250290191 >> 172.21.15.106:0/2250290191 conn(0x7f1fa009e3b0 msgr2=0x7f1fa00d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:59.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 -- 172.21.15.106:0/2250290191 shutdown_connections 2024-04-26T21:46:59.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.073+0000 7f1fa8800640 1 -- 172.21.15.106:0/2250290191 wait complete. 2024-04-26T21:46:59.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.075+0000 7f1fa8800640 1 Processor -- start 2024-04-26T21:46:59.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.075+0000 7f1fa8800640 1 -- start start 2024-04-26T21:46:59.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00baac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa015ef60 0x7f1fa00b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa00d0350 con 0x7f1fa00d5d90 2024-04-26T21:46:59.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa00d01d0 con 0x7f1fa015ef60 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa8800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa00d04d0 con 0x7f1fa00d34a0 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59652/0 (socket says 172.21.15.106:59652) 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.076+0000 7f1fa5400640 1 -- 172.21.15.106:0/1054885449 learned_addr learned my addr 172.21.15.106:0/1054885449 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 -- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 msgr2=0x7f1fa00baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 -- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa015ef60 msgr2=0x7f1fa00b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa015ef60 0x7f1fa00b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 -- 172.21.15.106:0/1054885449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa00cc9c0 con 0x7f1fa00d5d90 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa5400640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f900098c0 tx=0x7f1f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90037040 con 0x7f1fa00d5d90 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90030500 con 0x7f1fa00d5d90 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900308d0 con 0x7f1fa00d5d90 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa0152a90 con 0x7f1fa00d5d90 2024-04-26T21:46:59.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.077+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa00cc400 con 0x7f1fa00d5d90 2024-04-26T21:46:59.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.078+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa0069540 con 0x7f1fa00d5d90 2024-04-26T21:46:59.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.079+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f90044020 con 0x7f1fa00d5d90 2024-04-26T21:46:59.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.081+0000 7f1f9a000640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 0x7f1f8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.081+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1f900abf00 con 0x7f1fa00d5d90 2024-04-26T21:46:59.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.081+0000 7f1fa5e00640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 0x7f1f8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.082+0000 7f1fa5e00640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 0x7f1f8406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000f60 tx=0x7f1f940078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.085+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fa0152a90 con 0x7f1fa00d5d90 2024-04-26T21:46:59.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.270+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fa006d8f0 con 0x7f1fa00d5d90 2024-04-26T21:46:59.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.273+0000 7f1f9a000640 1 -- 172.21.15.106:0/1054885449 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f1fa00cc9c0 con 0x7f1fa00d5d90 2024-04-26T21:46:59.275 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:46:59.275 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 msgr2=0x7f1f8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 0x7f1f8406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000f60 tx=0x7f1f940078f0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 msgr2=0x7f1fa0152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f900098c0 tx=0x7f1f900046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 shutdown_connections 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa015ef60 0x7f1fa00b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fa00d5d90 0x7f1fa0152550 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f8406c8f0 0x7f1f8406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 --2- 172.21.15.106:0/1054885449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa00d34a0 0x7f1fa00baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 >> 172.21.15.106:0/1054885449 conn(0x7f1fa009e3b0 msgr2=0x7f1fa00b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:59.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 shutdown_connections 2024-04-26T21:46:59.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.276+0000 7f1fa8800640 1 -- 172.21.15.106:0/1054885449 wait complete. 2024-04-26T21:46:59.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:45:54.339744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:45:55.088283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:59.297 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.487+0000 7fe866600640 1 Processor -- start 2024-04-26T21:46:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.487+0000 7fe866600640 1 -- start start 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.488+0000 7fe866600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe860073470 0x7fe8600727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.488+0000 7fe866600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 0x7fe8600b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe866600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86006ea20 con 0x7fe8600b5ce0 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe866600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe86006e7d0 con 0x7fe860073470 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe866600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe860073120 con 0x7fe860072cf0 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe85fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe860073470 0x7fe8600727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe865200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe8600b5ce0 0x7fe86015ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40304/0 (socket says 172.21.15.106:40304) 2024-04-26T21:46:59.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe865200640 1 -- 172.21.15.106:0/2384167722 learned_addr learned my addr 172.21.15.106:0/2384167722 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:46:59.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe865c00640 1 --2- 172.21.15.106:0/2384167722 >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 0x7fe8600b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe85fe00640 1 -- 172.21.15.106:0/2384167722 >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 msgr2=0x7fe8600b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:59.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe85fe00640 1 --2- 172.21.15.106:0/2384167722 >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 0x7fe8600b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.489+0000 7fe85fe00640 1 -- 172.21.15.106:0/2384167722 >> v1:172.21.15.106:6789/0 conn(0x7fe8600b5ce0 legacy=0x7fe86015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.490+0000 7fe85fe00640 1 -- 172.21.15.106:0/2384167722 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86006ed00 con 0x7fe860073470 2024-04-26T21:46:59.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.490+0000 7fe85f400640 1 -- 172.21.15.106:0/2384167722 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4232822696 0 0) 0x7fe86006ea20 con 0x7fe8600b5ce0 2024-04-26T21:46:59.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.490+0000 7fe865c00640 1 --2- 172.21.15.106:0/2384167722 >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 0x7fe8600b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:59.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe85fe00640 1 --2- 172.21.15.106:0/2384167722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe860073470 0x7fe8600727b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe8600b69c0 tx=0x7fe84802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b071ad7d2ca55c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe85f400640 1 -- 172.21.15.106:0/2384167722 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848037070 con 0x7fe860073470 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe85f400640 1 -- 172.21.15.106:0/2384167722 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe848032440 con 0x7fe860073470 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe85f400640 1 -- 172.21.15.106:0/2384167722 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84802ae60 con 0x7fe860073470 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe866600640 1 -- 172.21.15.106:0/2384167722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe860073470 msgr2=0x7fe8600727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.491+0000 7fe866600640 1 --2- 172.21.15.106:0/2384167722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe860073470 0x7fe8600727b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe8600b69c0 tx=0x7fe84802a110 comp rx=0 tx=0).stop 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 -- 172.21.15.106:0/2384167722 shutdown_connections 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 --2- 172.21.15.106:0/2384167722 >> v2:172.21.15.204:3300/0 conn(0x7fe860072cf0 0x7fe8600b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 --2- 172.21.15.106:0/2384167722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe860073470 0x7fe8600727b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 -- 172.21.15.106:0/2384167722 >> 172.21.15.106:0/2384167722 conn(0x7fe860075f90 msgr2=0x7fe86006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:59.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 -- 172.21.15.106:0/2384167722 shutdown_connections 2024-04-26T21:46:59.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.492+0000 7fe866600640 1 -- 172.21.15.106:0/2384167722 wait complete. 2024-04-26T21:46:59.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.493+0000 7fe866600640 1 Processor -- start 2024-04-26T21:46:59.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 -- start start 2024-04-26T21:46:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe860072cf0 0x7fe86015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe860073470 0x7fe86015ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe8600b60c0 con 0x7fe8600b5ce0 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe86006f2c0 con 0x7fe860072cf0 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.494+0000 7fe866600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8600b6240 con 0x7fe860073470 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59680/0 (socket says 172.21.15.106:59680) 2024-04-26T21:46:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 -- 172.21.15.106:0/2657720975 learned_addr learned my addr 172.21.15.106:0/2657720975 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865200640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe860072cf0 0x7fe86015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 -- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe860073470 msgr2=0x7fe86015ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe860073470 0x7fe86015ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 -- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe860072cf0 msgr2=0x7fe86015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe860072cf0 0x7fe86015d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.495+0000 7fe865c00640 1 -- 172.21.15.106:0/2657720975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86015ddf0 con 0x7fe8600b5ce0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe865c00640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c5e0 tx=0x7fe85800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858015070 con 0x7fe8600b5ce0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe858011840 con 0x7fe8600b5ce0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86015e0d0 con 0x7fe8600b5ce0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858011c30 con 0x7fe8600b5ce0 2024-04-26T21:46:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.496+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86015e690 con 0x7fe8600b5ce0 2024-04-26T21:46:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.497+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe860069ad0 con 0x7fe8600b5ce0 2024-04-26T21:46:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.498+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe858018070 con 0x7fe8600b5ce0 2024-04-26T21:46:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.499+0000 7fe85cc00640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 0x7fe83c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.499+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe85808b9d0 con 0x7fe8600b5ce0 2024-04-26T21:46:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.499+0000 7fe865200640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 0x7fe83c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.500+0000 7fe865200640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 0x7fe83c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8500012e0 tx=0x7fe850009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.503+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe858059d70 con 0x7fe8600b5ce0 2024-04-26T21:46:59.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.707+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8600b4660 con 0x7fe8600b5ce0 2024-04-26T21:46:59.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.708+0000 7fe85cc00640 1 -- 172.21.15.106:0/2657720975 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe858059670 con 0x7fe8600b5ce0 2024-04-26T21:46:59.709 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.710+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 msgr2=0x7fe83c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.710+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 0x7fe83c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8500012e0 tx=0x7fe850009040 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.710+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 msgr2=0x7fe860148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.710+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c5e0 tx=0x7fe85800caa0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 shutdown_connections 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8600b5ce0 0x7fe860148e40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe83c06c920 0x7fe83c06ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe860073470 0x7fe86015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 --2- 172.21.15.106:0/2657720975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe860072cf0 0x7fe86015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 >> 172.21.15.106:0/2657720975 conn(0x7fe860075f90 msgr2=0x7fe8600b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:59.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 shutdown_connections 2024-04-26T21:46:59.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.711+0000 7fe866600640 1 -- 172.21.15.106:0/2657720975 wait complete. 2024-04-26T21:46:59.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:59.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.929+0000 7f0296c00640 1 Processor -- start 2024-04-26T21:46:59.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.929+0000 7f0296c00640 1 -- start start 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0296c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0290071e80 0x7f02900b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0296c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 0x7f02900ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0296c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029006e320 con 0x7f0290072260 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0296c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f029006e600 con 0x7f0290071e80 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0296c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f029006e8e0 con 0x7f02900b4ed0 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0295800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0290071e80 0x7f02900b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0295800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0290071e80 0x7f02900b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59684/0 (socket says 172.21.15.106:59684) 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.931+0000 7f0295800640 1 -- 172.21.15.106:0/4201958048 learned_addr learned my addr 172.21.15.106:0/4201958048 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0294e00640 1 --2- 172.21.15.106:0/4201958048 >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 0x7f02900ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0295800640 1 -- 172.21.15.106:0/4201958048 >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 msgr2=0x7f02900ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0295800640 1 --2- 172.21.15.106:0/4201958048 >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 0x7f02900ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0295800640 1 -- 172.21.15.106:0/4201958048 >> v1:172.21.15.204:6789/0 conn(0x7f0290072260 legacy=0x7f02900726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:59.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0295800640 1 -- 172.21.15.106:0/4201958048 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029006ebc0 con 0x7f0290071e80 2024-04-26T21:46:59.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.932+0000 7f0294e00640 1 --2- 172.21.15.106:0/4201958048 >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 0x7f02900ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.933+0000 7f0295800640 1 --2- 172.21.15.106:0/4201958048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f02900b3ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f02900cce70 tx=0x7f028402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfaf943ee1c9848 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.933+0000 7f028fe00640 1 -- 172.21.15.106:0/4201958048 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284037070 con 0x7f0290071e80 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.933+0000 7f028fe00640 1 -- 172.21.15.106:0/4201958048 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0284032440 con 0x7f0290071e80 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.933+0000 7f028fe00640 1 -- 172.21.15.106:0/4201958048 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284032810 con 0x7f0290071e80 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 -- 172.21.15.106:0/4201958048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 msgr2=0x7f02900b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 --2- 172.21.15.106:0/4201958048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f02900b3ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f02900cce70 tx=0x7f028402a230 comp rx=0 tx=0).stop 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 -- 172.21.15.106:0/4201958048 shutdown_connections 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 --2- 172.21.15.106:0/4201958048 >> v2:172.21.15.204:3300/0 conn(0x7f02900b4ed0 0x7f02900ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 --2- 172.21.15.106:0/4201958048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f02900b3ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 -- 172.21.15.106:0/4201958048 >> 172.21.15.106:0/4201958048 conn(0x7f02900b2900 msgr2=0x7f02900b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 -- 172.21.15.106:0/4201958048 shutdown_connections 2024-04-26T21:46:59.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.934+0000 7f0296c00640 1 -- 172.21.15.106:0/4201958048 wait complete. 2024-04-26T21:46:59.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.936+0000 7f0296c00640 1 Processor -- start 2024-04-26T21:46:59.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.936+0000 7f0296c00640 1 -- start start 2024-04-26T21:46:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0290072260 0x7f02901f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02900b4ed0 0x7f0290153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f029006f0e0 con 0x7f0290071e80 2024-04-26T21:46:59.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02900b5ff0 con 0x7f0290072260 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0296c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f029006f260 con 0x7f02900b4ed0 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59686/0 (socket says 172.21.15.106:59686) 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 -- 172.21.15.106:0/3573864455 learned_addr learned my addr 172.21.15.106:0/3573864455 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 -- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02900b4ed0 msgr2=0x7f0290153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02900b4ed0 0x7f0290153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 -- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0290072260 msgr2=0x7f02901f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0290072260 0x7f02901f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.937+0000 7f0295800640 1 -- 172.21.15.106:0/3573864455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0290150290 con 0x7f0290071e80 2024-04-26T21:46:59.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f0295800640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f028402a270 tx=0x7f02840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284037040 con 0x7f0290071e80 2024-04-26T21:46:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0284042070 con 0x7f0290071e80 2024-04-26T21:46:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0290150570 con 0x7f0290071e80 2024-04-26T21:46:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284030bd0 con 0x7f0290071e80 2024-04-26T21:46:59.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.938+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0290153250 con 0x7f0290071e80 2024-04-26T21:46:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.939+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02900693c0 con 0x7f0290071e80 2024-04-26T21:46:59.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.940+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0284047460 con 0x7f0290071e80 2024-04-26T21:46:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.941+0000 7f028d600640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.941+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f02840ac910 con 0x7f0290071e80 2024-04-26T21:46:59.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.941+0000 7f0294e00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.942+0000 7f0294e00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 0x7f027006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:46:59.945+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f028407ac20 con 0x7f0290071e80 2024-04-26T21:47:00.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.130+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02900af5d0 con 0x7f0290071e80 2024-04-26T21:47:00.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.131+0000 7f028d600640 1 -- 172.21.15.106:0/3573864455 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f028407a520 con 0x7f0290071e80 2024-04-26T21:47:00.132 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:47:00.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 msgr2=0x7f027006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 0x7f027006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 msgr2=0x7f0290152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f028402a270 tx=0x7f02840046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 shutdown_connections 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02900b4ed0 0x7f0290153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0290072260 0x7f02901f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 --2- 172.21.15.106:0/3573864455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0290071e80 0x7f0290152d10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.133+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 >> 172.21.15.106:0/3573864455 conn(0x7f02900b2900 msgr2=0x7f02900af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.134+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 shutdown_connections 2024-04-26T21:47:00.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.134+0000 7f0296c00640 1 -- 172.21.15.106:0/3573864455 wait complete. 2024-04-26T21:47:00.151 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:00.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.348+0000 7f4657600640 1 Processor -- start 2024-04-26T21:47:00.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.348+0000 7f4657600640 1 -- start start 2024-04-26T21:47:00.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.349+0000 7f4657600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f465015d100 0x7f465006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.349+0000 7f4657600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4657600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46500af800 con 0x7f46500b5ca0 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4657600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f46500ae340 con 0x7f465015d100 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4657600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46500afaa0 con 0x7f46500add80 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4654c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f465015d100 0x7f465006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4655600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4654c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f465015d100 0x7f465006c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:55236/0 (socket says 172.21.15.106:55236) 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4655600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:47928/0 (socket says 172.21.15.106:47928) 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4654c00640 1 -- 172.21.15.106:0/1697865431 learned_addr learned my addr 172.21.15.106:0/1697865431 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.350+0000 7f4647e00640 1 --1- 172.21.15.106:0/1697865431 >> v1:172.21.15.204:6789/0 conn(0x7f46500b5ca0 0x7f4650072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:42512/0 (socket says 172.21.15.106:42512) 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.351+0000 7f4654c00640 1 -- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 msgr2=0x7f465006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.351+0000 7f4654c00640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.351+0000 7f4654c00640 1 -- 172.21.15.106:0/1697865431 >> v1:172.21.15.204:6789/0 conn(0x7f46500b5ca0 legacy=0x7f4650072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.351+0000 7f4654c00640 1 -- 172.21.15.106:0/1697865431 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46500afd80 con 0x7f465015d100 2024-04-26T21:47:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.351+0000 7f4655600640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.352+0000 7f4654c00640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465015d100 0x7f465006c770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ae6039c7a5eb2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.353+0000 7f464fe00640 1 -- 172.21.15.106:0/1697865431 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638037070 con 0x7f465015d100 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.353+0000 7f464fe00640 1 -- 172.21.15.106:0/1697865431 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638032440 con 0x7f465015d100 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.353+0000 7f464fe00640 1 -- 172.21.15.106:0/1697865431 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638032810 con 0x7f465015d100 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 -- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465015d100 msgr2=0x7f465006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465015d100 0x7f465006c770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 -- 172.21.15.106:0/1697865431 shutdown_connections 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500add80 0x7f465006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 --2- 172.21.15.106:0/1697865431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465015d100 0x7f465006c770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 -- 172.21.15.106:0/1697865431 >> 172.21.15.106:0/1697865431 conn(0x7f46500b2d50 msgr2=0x7f46500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 -- 172.21.15.106:0/1697865431 shutdown_connections 2024-04-26T21:47:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.354+0000 7f4657600640 1 -- 172.21.15.106:0/1697865431 wait complete. 2024-04-26T21:47:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.356+0000 7f4657600640 1 Processor -- start 2024-04-26T21:47:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.356+0000 7f4657600640 1 -- start start 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46500b5ca0 0x7f46501f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f465015d100 0x7f46501f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f465009ee50 con 0x7f46500add80 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46500b0340 con 0x7f46500b5ca0 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4657600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4650160b70 con 0x7f465015d100 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4654c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4654c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59694/0 (socket says 172.21.15.106:59694) 2024-04-26T21:47:00.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.357+0000 7f4654c00640 1 -- 172.21.15.106:0/3748060560 learned_addr learned my addr 172.21.15.106:0/3748060560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 -- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f465015d100 msgr2=0x7f46501f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f465015d100 0x7f46501f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 -- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46500b5ca0 msgr2=0x7f46501f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46500b5ca0 0x7f46501f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 -- 172.21.15.106:0/3748060560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465006eba0 con 0x7f46500add80 2024-04-26T21:47:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f4654c00640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638037040 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638042070 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.358+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803b870 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.359+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465006ee80 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.359+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465006f440 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.360+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4650161a10 con 0x7f46500add80 2024-04-26T21:47:00.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.361+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f463802ad10 con 0x7f46500add80 2024-04-26T21:47:00.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.362+0000 7f464d600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 0x7f462c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.362+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f46380ac080 con 0x7f46500add80 2024-04-26T21:47:00.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.362+0000 7f4647e00640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 0x7f462c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.363+0000 7f4647e00640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 0x7f462c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4640000f60 tx=0x7f4640007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.367+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4650161a10 con 0x7f46500add80 2024-04-26T21:47:00.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.565+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4650069650 con 0x7f46500add80 2024-04-26T21:47:00.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.566+0000 7f464d600640 1 -- 172.21.15.106:0/3748060560 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f463807a430 con 0x7f46500add80 2024-04-26T21:47:00.567 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.568+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 msgr2=0x7f462c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.568+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 0x7f462c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4640000f60 tx=0x7f4640007a00 comp rx=0 tx=0).stop 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.568+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 msgr2=0x7f465009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.568+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 shutdown_connections 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f462c06c8f0 0x7f462c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f465015d100 0x7f46501f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46500b5ca0 0x7f46501f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 --2- 172.21.15.106:0/3748060560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46500add80 0x7f465009ba50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 >> 172.21.15.106:0/3748060560 conn(0x7f46500b2d50 msgr2=0x7f46500b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 shutdown_connections 2024-04-26T21:47:00.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.569+0000 7f4657600640 1 -- 172.21.15.106:0/3748060560 wait complete. 2024-04-26T21:47:00.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:00.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.789+0000 7fd03ba00640 1 Processor -- start 2024-04-26T21:47:00.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.789+0000 7fd03ba00640 1 -- start start 2024-04-26T21:47:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd03ba00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd03415cdf0 0x7fd0340add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd03ba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd0340cd650 0x7fd03406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd03ba00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0340af9a0 con 0x7fd0340b4f50 2024-04-26T21:47:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd03ba00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd0340ae2c0 con 0x7fd03415cdf0 2024-04-26T21:47:00.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd03ba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd0340b05b0 con 0x7fd0340cd650 2024-04-26T21:47:00.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd039a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd0340b4f50 0x7fd034072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40306/0 (socket says 172.21.15.106:40306) 2024-04-26T21:47:00.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd039a00640 1 -- 172.21.15.106:0/1463685841 learned_addr learned my addr 172.21.15.106:0/1463685841 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:00.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.791+0000 7fd039000640 1 --2- 172.21.15.106:0/1463685841 >> v2:172.21.15.106:3300/0 conn(0x7fd03415cdf0 0x7fd0340add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd02be00640 1 --2- 172.21.15.106:0/1463685841 >> v2:172.21.15.204:3300/0 conn(0x7fd0340cd650 0x7fd03406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd033e00640 1 -- 172.21.15.106:0/1463685841 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947400316 0 0) 0x7fd0340af9a0 con 0x7fd0340b4f50 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd033e00640 1 -- 172.21.15.106:0/1463685841 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd018003520 con 0x7fd0340b4f50 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd039000640 1 -- 172.21.15.106:0/1463685841 >> v2:172.21.15.204:3300/0 conn(0x7fd0340cd650 msgr2=0x7fd03406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd039000640 1 --2- 172.21.15.106:0/1463685841 >> v2:172.21.15.204:3300/0 conn(0x7fd0340cd650 0x7fd03406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd039000640 1 -- 172.21.15.106:0/1463685841 >> v1:172.21.15.106:6789/0 conn(0x7fd0340b4f50 legacy=0x7fd034072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.792+0000 7fd039000640 1 -- 172.21.15.106:0/1463685841 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0340afc80 con 0x7fd03415cdf0 2024-04-26T21:47:00.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.793+0000 7fd039000640 1 --2- 172.21.15.106:0/1463685841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd0340add80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd0340d1d00 tx=0x7fd02402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b6e8febe1c3ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.793+0000 7fd033e00640 1 -- 172.21.15.106:0/1463685841 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024037070 con 0x7fd03415cdf0 2024-04-26T21:47:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.793+0000 7fd033e00640 1 -- 172.21.15.106:0/1463685841 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd024032440 con 0x7fd03415cdf0 2024-04-26T21:47:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.793+0000 7fd033e00640 1 -- 172.21.15.106:0/1463685841 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024032810 con 0x7fd03415cdf0 2024-04-26T21:47:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 -- 172.21.15.106:0/1463685841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 msgr2=0x7fd0340add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 --2- 172.21.15.106:0/1463685841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd0340add80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd0340d1d00 tx=0x7fd02402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 -- 172.21.15.106:0/1463685841 shutdown_connections 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 --2- 172.21.15.106:0/1463685841 >> v2:172.21.15.204:3300/0 conn(0x7fd0340cd650 0x7fd03406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 --2- 172.21.15.106:0/1463685841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd0340add80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 -- 172.21.15.106:0/1463685841 >> 172.21.15.106:0/1463685841 conn(0x7fd0340b2d50 msgr2=0x7fd0340b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 -- 172.21.15.106:0/1463685841 shutdown_connections 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.794+0000 7fd03ba00640 1 -- 172.21.15.106:0/1463685841 wait complete. 2024-04-26T21:47:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.796+0000 7fd03ba00640 1 Processor -- start 2024-04-26T21:47:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.796+0000 7fd03ba00640 1 -- start start 2024-04-26T21:47:00.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340b4f50 0x7fd03414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0340cd650 0x7fd034158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd03415d1d0 con 0x7fd03415cdf0 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0340b0240 con 0x7fd0340b4f50 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd03ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd03415d350 con 0x7fd0340cd650 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59706/0 (socket says 172.21.15.106:59706) 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 -- 172.21.15.106:0/4085885434 learned_addr learned my addr 172.21.15.106:0/4085885434 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039000640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340b4f50 0x7fd03414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd02be00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0340cd650 0x7fd034158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 -- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0340cd650 msgr2=0x7fd034158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0340cd650 0x7fd034158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 -- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340b4f50 msgr2=0x7fd03414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340b4f50 0x7fd03414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.797+0000 7fd039a00640 1 -- 172.21.15.106:0/4085885434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03409c710 con 0x7fd03415cdf0 2024-04-26T21:47:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd039a00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00cb40 tx=0x7fd02c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c017070 con 0x7fd03415cdf0 2024-04-26T21:47:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c00aca0 con 0x7fd03415cdf0 2024-04-26T21:47:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c012620 con 0x7fd03415cdf0 2024-04-26T21:47:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03409c9f0 con 0x7fd03415cdf0 2024-04-26T21:47:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.798+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03409cfb0 con 0x7fd03415cdf0 2024-04-26T21:47:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.799+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd034069540 con 0x7fd03415cdf0 2024-04-26T21:47:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.800+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd02c01a070 con 0x7fd03415cdf0 2024-04-26T21:47:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.801+0000 7fd031600640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.801+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd02c08ba70 con 0x7fd03415cdf0 2024-04-26T21:47:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.801+0000 7fd039000640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:00.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.802+0000 7fd039000640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 0x7fd01006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd024034000 tx=0x7fd02402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:00.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:00.805+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd02c059d80 con 0x7fd03415cdf0 2024-04-26T21:47:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.046+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0340d13c0 con 0x7fd03415cdf0 2024-04-26T21:47:01.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.047+0000 7fd031600640 1 -- 172.21.15.106:0/4085885434 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7fd02c059680 con 0x7fd03415cdf0 2024-04-26T21:47:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:01.048 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:47:01.048 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:47:01.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 msgr2=0x7fd01006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 0x7fd01006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd024034000 tx=0x7fd02402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:01.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 msgr2=0x7fd034158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00cb40 tx=0x7fd02c00a060 comp rx=0 tx=0).stop 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 shutdown_connections 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd03415cdf0 0x7fd034158580 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0340cd650 0x7fd034158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 --2- 172.21.15.106:0/4085885434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340b4f50 0x7fd03414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 >> 172.21.15.106:0/4085885434 conn(0x7fd0340b2d50 msgr2=0x7fd0340b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 shutdown_connections 2024-04-26T21:47:01.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.049+0000 7fd03ba00640 1 -- 172.21.15.106:0/4085885434 wait complete. 2024-04-26T21:47:01.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:01.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.258+0000 7fa047800640 1 Processor -- start 2024-04-26T21:47:01.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.258+0000 7fa047800640 1 -- start start 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa047800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa04015cdf0 0x7fa0400add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa047800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa04006ce80 0x7fa0400b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa047800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0400af9a0 con 0x7fa0400cd650 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa047800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa0400ae2c0 con 0x7fa04015cdf0 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa047800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa0400b05b0 con 0x7fa04006ce80 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa044e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa04015cdf0 0x7fa0400add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa03fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa0400cd650 0x7fa04006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40316/0 (socket says 172.21.15.106:40316) 2024-04-26T21:47:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.260+0000 7fa03fe00640 1 -- 172.21.15.106:0/1044585594 learned_addr learned my addr 172.21.15.106:0/1044585594 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:01.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa045800640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa04006ce80 0x7fa0400b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa03f400640 1 -- 172.21.15.106:0/1044585594 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557332186 0 0) 0x7fa0400af9a0 con 0x7fa0400cd650 2024-04-26T21:47:01.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa03f400640 1 -- 172.21.15.106:0/1044585594 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa028003520 con 0x7fa0400cd650 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa044e00640 1 -- 172.21.15.106:0/1044585594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa04006ce80 msgr2=0x7fa0400b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa044e00640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa04006ce80 0x7fa0400b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa044e00640 1 -- 172.21.15.106:0/1044585594 >> v1:172.21.15.106:6789/0 conn(0x7fa0400cd650 legacy=0x7fa04006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.261+0000 7fa044e00640 1 -- 172.21.15.106:0/1044585594 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400afc80 con 0x7fa04015cdf0 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.262+0000 7fa044e00640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04015cdf0 0x7fa0400add80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa0340097a0 tx=0x7fa03402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb0131bc4aef44d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa03f400640 1 -- 172.21.15.106:0/1044585594 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034037070 con 0x7fa04015cdf0 2024-04-26T21:47:01.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa03f400640 1 -- 172.21.15.106:0/1044585594 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa034032440 con 0x7fa04015cdf0 2024-04-26T21:47:01.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa03f400640 1 -- 172.21.15.106:0/1044585594 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0340327f0 con 0x7fa04015cdf0 2024-04-26T21:47:01.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 -- 172.21.15.106:0/1044585594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04015cdf0 msgr2=0x7fa0400add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04015cdf0 0x7fa0400add80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa0340097a0 tx=0x7fa03402a180 comp rx=0 tx=0).stop 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 -- 172.21.15.106:0/1044585594 shutdown_connections 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa04006ce80 0x7fa0400b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 --2- 172.21.15.106:0/1044585594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04015cdf0 0x7fa0400add80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.263+0000 7fa047800640 1 -- 172.21.15.106:0/1044585594 >> 172.21.15.106:0/1044585594 conn(0x7fa0400b2d50 msgr2=0x7fa0400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.264+0000 7fa047800640 1 -- 172.21.15.106:0/1044585594 shutdown_connections 2024-04-26T21:47:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.264+0000 7fa047800640 1 -- 172.21.15.106:0/1044585594 wait complete. 2024-04-26T21:47:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.266+0000 7fa047800640 1 Processor -- start 2024-04-26T21:47:01.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.266+0000 7fa047800640 1 -- start start 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0400cd650 0x7fa040158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa04015cdf0 0x7fa0401588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa04015d1d0 con 0x7fa04006ce80 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400b0240 con 0x7fa04015cdf0 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa047800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa04015d350 con 0x7fa0400cd650 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa044e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa044e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59724/0 (socket says 172.21.15.106:59724) 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.267+0000 7fa044e00640 1 -- 172.21.15.106:0/1152411801 learned_addr learned my addr 172.21.15.106:0/1152411801 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa03fe00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0400cd650 0x7fa040158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 -- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0400cd650 msgr2=0x7fa040158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0400cd650 0x7fa040158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 -- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa04015cdf0 msgr2=0x7fa0401588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa04015cdf0 0x7fa0401588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 -- 172.21.15.106:0/1152411801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04006f2d0 con 0x7fa04006ce80 2024-04-26T21:47:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa044e00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa034035040 tx=0x7fa0340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.268+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034037040 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.269+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0340044c0 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.269+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04006f5b0 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.269+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0340325f0 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.269+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04006fb70 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.270+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa040069540 con 0x7fa04006ce80 2024-04-26T21:47:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.271+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa034044020 con 0x7fa04006ce80 2024-04-26T21:47:01.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.272+0000 7fa03cc00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.272+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa0340abf00 con 0x7fa04006ce80 2024-04-26T21:47:01.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.272+0000 7fa03fe00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.273+0000 7fa03fe00640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0401596b0 tx=0x7fa02c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.276+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa03407a2b0 con 0x7fa04006ce80 2024-04-26T21:47:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.477+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0400b21c0 con 0x7fa04006ce80 2024-04-26T21:47:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.480+0000 7fa03cc00640 1 -- 172.21.15.106:0/1152411801 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7fa034079bb0 con 0x7fa04006ce80 2024-04-26T21:47:01.481 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:01.481 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.482+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 msgr2=0x7fa02006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.482+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0401596b0 tx=0x7fa02c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.482+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 msgr2=0x7fa04015dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.482+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa034035040 tx=0x7fa0340046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 shutdown_connections 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa04015cdf0 0x7fa0401588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0400cd650 0x7fa040158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 --2- 172.21.15.106:0/1152411801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa04006ce80 0x7fa04015dfe0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 >> 172.21.15.106:0/1152411801 conn(0x7fa0400b2d50 msgr2=0x7fa04006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 shutdown_connections 2024-04-26T21:47:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.483+0000 7fa047800640 1 -- 172.21.15.106:0/1152411801 wait complete. 2024-04-26T21:47:01.503 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:47:01.503 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:01.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.701+0000 7f9a21400640 1 Processor -- start 2024-04-26T21:47:01.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.701+0000 7f9a21400640 1 -- start start 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c15cdf0 0x7f9a1c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 0x7f9a1c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a1c0b4f50 0x7f9a1c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c0ae2c0 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c0b05b0 con 0x7f9a1c0b4f50 2024-04-26T21:47:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a21400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0af9a0 con 0x7f9a1c0cd650 2024-04-26T21:47:01.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a1a800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c15cdf0 0x7f9a1c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a1a800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c15cdf0 0x7f9a1c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59736/0 (socket says 172.21.15.106:59736) 2024-04-26T21:47:01.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.703+0000 7f9a1a800640 1 -- 172.21.15.106:0/2038456109 learned_addr learned my addr 172.21.15.106:0/2038456109 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:01.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a11e00640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 0x7f9a1c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1b200640 1 --2- 172.21.15.106:0/2038456109 >> v2:172.21.15.204:3300/0 conn(0x7f9a1c0b4f50 0x7f9a1c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1a800640 1 -- 172.21.15.106:0/2038456109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 msgr2=0x7f9a1c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1a800640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 0x7f9a1c06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1a800640 1 -- 172.21.15.106:0/2038456109 >> v2:172.21.15.204:3300/0 conn(0x7f9a1c0b4f50 msgr2=0x7f9a1c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1a800640 1 --2- 172.21.15.106:0/2038456109 >> v2:172.21.15.204:3300/0 conn(0x7f9a1c0b4f50 0x7f9a1c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a1a800640 1 -- 172.21.15.106:0/2038456109 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c0afc80 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.704+0000 7f9a11e00640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 0x7f9a1c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:01.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.705+0000 7f9a1a800640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c0add80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0b0520 tx=0x7f9a0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f661d68dc9167de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.706+0000 7f9a19e00640 1 -- 172.21.15.106:0/2038456109 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c037070 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.706+0000 7f9a19e00640 1 -- 172.21.15.106:0/2038456109 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c032440 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.706+0000 7f9a19e00640 1 -- 172.21.15.106:0/2038456109 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c032810 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.706+0000 7f9a21400640 1 -- 172.21.15.106:0/2038456109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 msgr2=0x7f9a1c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.706+0000 7f9a21400640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c0add80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0b0520 tx=0x7f9a0c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 -- 172.21.15.106:0/2038456109 shutdown_connections 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 --2- 172.21.15.106:0/2038456109 >> v2:172.21.15.204:3300/0 conn(0x7f9a1c0b4f50 0x7f9a1c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0cd650 0x7f9a1c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 --2- 172.21.15.106:0/2038456109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c0add80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 -- 172.21.15.106:0/2038456109 >> 172.21.15.106:0/2038456109 conn(0x7f9a1c0b2d50 msgr2=0x7f9a1c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 -- 172.21.15.106:0/2038456109 shutdown_connections 2024-04-26T21:47:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.707+0000 7f9a21400640 1 -- 172.21.15.106:0/2038456109 wait complete. 2024-04-26T21:47:01.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.708+0000 7f9a21400640 1 Processor -- start 2024-04-26T21:47:01.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.709+0000 7f9a21400640 1 -- start start 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.709+0000 7f9a21400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0b4f50 0x7f9a1c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.709+0000 7f9a21400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 0x7f9a1c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a21400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a21400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c15d1d0 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a21400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c0b0240 con 0x7f9a1c0cd650 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a21400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c15d350 con 0x7f9a1c0b4f50 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59738/0 (socket says 172.21.15.106:59738) 2024-04-26T21:47:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 -- 172.21.15.106:0/426296096 learned_addr learned my addr 172.21.15.106:0/426296096 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a11e00640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 0x7f9a1c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1a800640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0b4f50 0x7f9a1c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 -- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0b4f50 msgr2=0x7f9a1c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0b4f50 0x7f9a1c156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 -- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 msgr2=0x7f9a1c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 0x7f9a1c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a1b200640 1 -- 172.21.15.106:0/426296096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c1f6050 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.710+0000 7f9a11e00640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 0x7f9a1c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a1b200640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9a14007e60 tx=0x7f9a1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14014420 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14019070 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14014820 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c1f6330 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.711+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1f68f0 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.712+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c1f6de0 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.713+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a140149c0 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.714+0000 7f9a13400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:01.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.715+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9a1408c320 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.715+0000 7f9a1a800640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.715+0000 7f9a1a800640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c034000 tx=0x7f9a0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:01.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.721+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a1405a760 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.959+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a1c0b0970 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.960+0000 7f9a13400640 1 -- 172.21.15.106:0/426296096 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f9a1405a060 con 0x7f9a1c15cdf0 2024-04-26T21:47:01.962 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:01.962 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:47:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:47:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 msgr2=0x7f99f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c034000 tx=0x7f9a0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 msgr2=0x7f9a1c15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9a14007e60 tx=0x7f9a1400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 shutdown_connections 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c15cdf0 0x7f9a1c15b4d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c0cd650 0x7f9a1c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 --2- 172.21.15.106:0/426296096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0b4f50 0x7f9a1c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 >> 172.21.15.106:0/426296096 conn(0x7f9a1c0b2d50 msgr2=0x7f9a1c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 shutdown_connections 2024-04-26T21:47:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:01.963+0000 7f9a21400640 1 -- 172.21.15.106:0/426296096 wait complete. 2024-04-26T21:47:01.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:02.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.176+0000 7ff637200640 1 Processor -- start 2024-04-26T21:47:02.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.176+0000 7ff637200640 1 -- start start 2024-04-26T21:47:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff637200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff6300add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff637200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6300af9a0 con 0x7ff63006ce80 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff637200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6300afc80 con 0x7ff63015cdf0 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff637200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff6300ae2c0 con 0x7ff6300cd650 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff62f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff63006ce80 0x7ff6300b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40320/0 (socket says 172.21.15.106:40320) 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.178+0000 7ff62f400640 1 -- 172.21.15.106:0/2952028124 learned_addr learned my addr 172.21.15.106:0/2952028124 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff635200640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff6300add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532007915 0 0) 0x7ff6300af9a0 con 0x7ff63006ce80 2024-04-26T21:47:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff614003520 con 0x7ff63006ce80 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1911283734 0 0) 0x7ff614003520 con 0x7ff63006ce80 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 msgr2=0x7ff6300add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff6300add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 >> v1:172.21.15.204:6789/0 conn(0x7ff63015cdf0 legacy=0x7ff63006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300aff60 con 0x7ff63006ce80 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7ff61c001670 con 0x7ff63006ce80 2024-04-26T21:47:02.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 >> v1:172.21.15.106:6789/0 conn(0x7ff63006ce80 legacy=0x7ff6300b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6140039e0 0x7ff614003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff614004d40 0x7ff6140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff61400ba70 con 0x7ff6140039e0 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff61400bc20 con 0x7ff614004d40 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff61400be20 con 0x7ff6300cd650 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.179+0000 7ff62fe00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6140039e0 0x7ff614003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff635200640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff614004d40 0x7ff6140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62f400640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62fe00640 1 -- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 msgr2=0x7ff614004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62fe00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff614004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62fe00640 1 -- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff614004d40 msgr2=0x7ff6140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62fe00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff614004d40 0x7ff6140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62fe00640 1 -- 172.21.15.106:0/2952028124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff614003520 con 0x7ff6140039e0 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff62f400640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6300cd650 0x7ff614004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.180+0000 7ff635200640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff614004d40 0x7ff6140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.181+0000 7ff62fe00640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6140039e0 0x7ff614003e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff620001720 tx=0x7ff614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3df7110d575279c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.181+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620001370 con 0x7ff6140039e0 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.181+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6200052d0 con 0x7ff6140039e0 2024-04-26T21:47:02.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.181+0000 7ff62ea00640 1 -- 172.21.15.106:0/2952028124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62002abc0 con 0x7ff6140039e0 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 -- 172.21.15.106:0/2952028124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6140039e0 msgr2=0x7ff614003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 --2- 172.21.15.106:0/2952028124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6140039e0 0x7ff614003e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff620001720 tx=0x7ff614005830 comp rx=0 tx=0).stop 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff635200640 1 -- 172.21.15.106:0/2952028124 reap_dead start 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 -- 172.21.15.106:0/2952028124 shutdown_connections 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 -- 172.21.15.106:0/2952028124 >> 172.21.15.106:0/2952028124 conn(0x7ff6300b2d50 msgr2=0x7ff6300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 -- 172.21.15.106:0/2952028124 shutdown_connections 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.182+0000 7ff637200640 1 -- 172.21.15.106:0/2952028124 wait complete. 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.184+0000 7ff637200640 1 Processor -- start 2024-04-26T21:47:02.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.184+0000 7ff637200640 1 -- start start 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6140039e0 0x7ff63015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff630150490 0x7ff63006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff63015d1d0 con 0x7ff63006f6d0 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300b05b0 con 0x7ff630150490 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff637200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff63015d350 con 0x7ff6140039e0 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff635200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff635200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59752/0 (socket says 172.21.15.106:59752) 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff635200640 1 -- 172.21.15.106:0/1125083174 learned_addr learned my addr 172.21.15.106:0/1125083174 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff62fe00640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6140039e0 0x7ff63015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff635200640 1 -- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6140039e0 msgr2=0x7ff63015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.185+0000 7ff635200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6140039e0 0x7ff63015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff635200640 1 -- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff630150490 msgr2=0x7ff63006f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff635200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff630150490 0x7ff63006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff635200640 1 -- 172.21.15.106:0/1125083174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300ce4b0 con 0x7ff63006f6d0 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff635200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6300d0380 tx=0x7ff62800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62801c070 con 0x7ff63006f6d0 2024-04-26T21:47:02.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628017400 con 0x7ff63006f6d0 2024-04-26T21:47:02.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63009d190 con 0x7ff63006f6d0 2024-04-26T21:47:02.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280177d0 con 0x7ff63006f6d0 2024-04-26T21:47:02.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.186+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6300cea30 con 0x7ff63006f6d0 2024-04-26T21:47:02.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.187+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff630069540 con 0x7ff63006f6d0 2024-04-26T21:47:02.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.188+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff62801f070 con 0x7ff63006f6d0 2024-04-26T21:47:02.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.189+0000 7ff627e00640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.189+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff62808bb40 con 0x7ff63006f6d0 2024-04-26T21:47:02.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.189+0000 7ff62fe00640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.190+0000 7ff62fe00640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 0x7ff61006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff620032000 tx=0x7ff62002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.194+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff628012070 con 0x7ff63006f6d0 2024-04-26T21:47:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.437+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff630151e50 con 0x7ff63006f6d0 2024-04-26T21:47:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.438+0000 7ff627e00640 1 -- 172.21.15.106:0/1125083174 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7ff628059f80 con 0x7ff63006f6d0 2024-04-26T21:47:02.439 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:02.439 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4077542834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4077542834},{"type":"v1","addr":"172.21.15.106:6837","nonce":4077542834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1532419858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1532419858},{"type":"v1","addr":"172.21.15.106:6835","nonce":1532419858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/514437626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":514437626},{"type":"v1","addr":"172.21.15.204:6833","nonce":514437626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:45:56.102085+0000","modified":"2024-04-26T21:45:59.159592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34166":{"gid":34166,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4043819607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":4043819607},{"type":"v1","addr":"172.21.15.204:6835","nonce":4043819607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34166,"qdb_cluster":[34166]},"id":1}]} 2024-04-26T21:47:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 msgr2=0x7ff61006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 0x7ff61006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff620032000 tx=0x7ff62002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 msgr2=0x7ff63009c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6300d0380 tx=0x7ff62800f4a0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 shutdown_connections 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63006f6d0 0x7ff63009c460 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff630150490 0x7ff63006f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 --2- 172.21.15.106:0/1125083174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6140039e0 0x7ff63015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 >> 172.21.15.106:0/1125083174 conn(0x7ff6300b2d50 msgr2=0x7ff63015c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 shutdown_connections 2024-04-26T21:47:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.441+0000 7ff637200640 1 -- 172.21.15.106:0/1125083174 wait complete. 2024-04-26T21:47:02.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:02.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.661+0000 7f4b5f200640 1 Processor -- start 2024-04-26T21:47:02.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.662+0000 7f4b5f200640 1 -- start start 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5f200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b600d5990 0x7f4b60163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5f200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b600d3f50 con 0x7f4b6015f1b0 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b600d36b0 con 0x7f4b600b82b0 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5f200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4b600d3990 con 0x7f4b600d5990 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5de00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4b6015f1b0 0x7f4b60162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40336/0 (socket says 172.21.15.106:40336) 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5de00640 1 -- 172.21.15.106:0/3080593068 learned_addr learned my addr 172.21.15.106:0/3080593068 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:02.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.663+0000 7f4b5d400640 1 --2- 172.21.15.106:0/3080593068 >> v2:172.21.15.106:3300/0 conn(0x7f4b600d5990 0x7f4b60163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.664+0000 7f4b5ca00640 1 -- 172.21.15.106:0/3080593068 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306793111 0 0) 0x7f4b600d3f50 con 0x7f4b6015f1b0 2024-04-26T21:47:02.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.664+0000 7f4b5ca00640 1 -- 172.21.15.106:0/3080593068 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b44003520 con 0x7f4b6015f1b0 2024-04-26T21:47:02.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.664+0000 7f4b5d400640 1 -- 172.21.15.106:0/3080593068 >> v1:172.21.15.204:6789/0 conn(0x7f4b600b82b0 legacy=0x7f4b600b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:02.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.664+0000 7f4b5d400640 1 -- 172.21.15.106:0/3080593068 >> v1:172.21.15.106:6789/0 conn(0x7f4b6015f1b0 legacy=0x7f4b60162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.664+0000 7f4b5d400640 1 -- 172.21.15.106:0/3080593068 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b600d3c70 con 0x7f4b600d5990 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.665+0000 7f4b5d400640 1 --2- 172.21.15.106:0/3080593068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b600d5990 0x7f4b60163210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4b480097d0 tx=0x7f4b4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4c07199f9e2449 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.665+0000 7f4b5ca00640 1 -- 172.21.15.106:0/3080593068 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48037070 con 0x7f4b600d5990 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.665+0000 7f4b5ca00640 1 -- 172.21.15.106:0/3080593068 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48032440 con 0x7f4b600d5990 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.665+0000 7f4b5ca00640 1 -- 172.21.15.106:0/3080593068 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48032810 con 0x7f4b600d5990 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 -- 172.21.15.106:0/3080593068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b600d5990 msgr2=0x7f4b60163210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 --2- 172.21.15.106:0/3080593068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b600d5990 0x7f4b60163210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4b480097d0 tx=0x7f4b4802a210 comp rx=0 tx=0).stop 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 -- 172.21.15.106:0/3080593068 shutdown_connections 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 --2- 172.21.15.106:0/3080593068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b600d5990 0x7f4b60163210 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 -- 172.21.15.106:0/3080593068 >> 172.21.15.106:0/3080593068 conn(0x7f4b6009e230 msgr2=0x7f4b600d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:02.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 -- 172.21.15.106:0/3080593068 shutdown_connections 2024-04-26T21:47:02.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.666+0000 7f4b5f200640 1 -- 172.21.15.106:0/3080593068 wait complete. 2024-04-26T21:47:02.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.667+0000 7f4b5f200640 1 Processor -- start 2024-04-26T21:47:02.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.668+0000 7f4b5f200640 1 -- start start 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.668+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b600b82b0 0x7f4b600c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.668+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b600d5990 0x7f4b600cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.668+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60163750 con 0x7f4b6015f1b0 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b600d33a0 con 0x7f4b600d5990 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b601638d0 con 0x7f4b600b82b0 2024-04-26T21:47:02.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59774/0 (socket says 172.21.15.106:59774) 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 -- 172.21.15.106:0/2084556881 learned_addr learned my addr 172.21.15.106:0/2084556881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5d400640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b600d5990 0x7f4b600cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 -- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b600b82b0 msgr2=0x7f4b600c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b600b82b0 0x7f4b600c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 -- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b600d5990 msgr2=0x7f4b600cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b600d5990 0x7f4b600cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.669+0000 7f4b5e800640 1 -- 172.21.15.106:0/2084556881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b600c8030 con 0x7f4b6015f1b0 2024-04-26T21:47:02.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b5e800640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c780 tx=0x7f4b5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58004b60 con 0x7f4b6015f1b0 2024-04-26T21:47:02.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b58004d00 con 0x7f4b6015f1b0 2024-04-26T21:47:02.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b600cc7e0 con 0x7f4b6015f1b0 2024-04-26T21:47:02.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800aa90 con 0x7f4b6015f1b0 2024-04-26T21:47:02.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.670+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b600ccda0 con 0x7f4b6015f1b0 2024-04-26T21:47:02.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.671+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b600693c0 con 0x7f4b6015f1b0 2024-04-26T21:47:02.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.672+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b5800ac30 con 0x7f4b6015f1b0 2024-04-26T21:47:02.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.673+0000 7f4b56000640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 0x7f4b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:02.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.673+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4b5808bf10 con 0x7f4b6015f1b0 2024-04-26T21:47:02.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.673+0000 7f4b5de00640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 0x7f4b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:02.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.674+0000 7f4b5de00640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 0x7f4b3c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b50000f60 tx=0x7f4b500078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.677+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b58020070 con 0x7f4b6015f1b0 2024-04-26T21:47:02.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.878+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b600d3f50 con 0x7f4b6015f1b0 2024-04-26T21:47:02.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.879+0000 7f4b56000640 1 -- 172.21.15.106:0/2084556881 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16810 (secure 0 0 0) 0x7f4b600c8030 con 0x7f4b6015f1b0 2024-04-26T21:47:02.881 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:02.881 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:46:38.916939+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:45:54.339744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:45:55.088283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.882+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 msgr2=0x7f4b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.882+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 0x7f4b3c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b50000f60 tx=0x7f4b500078f0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.882+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 msgr2=0x7f4b60158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.882+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c780 tx=0x7f4b5800cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 shutdown_connections 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b6015f1b0 0x7f4b60158d20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b600d5990 0x7f4b600cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4b3c06c8f0 0x7f4b3c06edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 --2- 172.21.15.106:0/2084556881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b600b82b0 0x7f4b600c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:02.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 >> 172.21.15.106:0/2084556881 conn(0x7f4b6009e230 msgr2=0x7f4b6015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:02.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 shutdown_connections 2024-04-26T21:47:02.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:02.883+0000 7f4b5f200640 1 -- 172.21.15.106:0/2084556881 wait complete. 2024-04-26T21:47:02.902 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:47:03.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.103+0000 7f74bfa00640 1 Processor -- start 2024-04-26T21:47:03.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.104+0000 7f74bfa00640 1 -- start start 2024-04-26T21:47:03.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.105+0000 7f74bfa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74b806ce80 0x7f74b81638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.105+0000 7f74bfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b815fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.105+0000 7f74bfa00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b80b5fb0 con 0x7f74b81598a0 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.105+0000 7f74bfa00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f74b8159c80 con 0x7f74b806ce80 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.105+0000 7f74bfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f74b81600f0 con 0x7f74b815f740 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74b806ce80 0x7f74b81638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74bd000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f74b81598a0 0x7f74b806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40340/0 (socket says 172.21.15.106:40340) 2024-04-26T21:47:03.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74bd000640 1 -- 172.21.15.106:0/1510303513 learned_addr learned my addr 172.21.15.106:0/1510303513 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74bda00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b815fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7400640 1 -- 172.21.15.106:0/1510303513 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4216429894 0 0) 0x7f74b80b5fb0 con 0x7f74b81598a0 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7400640 1 -- 172.21.15.106:0/1510303513 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7498003520 con 0x7f74b81598a0 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7e00640 1 -- 172.21.15.106:0/1510303513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 msgr2=0x7f74b815fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7e00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b815fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7e00640 1 -- 172.21.15.106:0/1510303513 >> v1:172.21.15.106:6789/0 conn(0x7f74b81598a0 legacy=0x7f74b806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74b7e00640 1 -- 172.21.15.106:0/1510303513 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b8160370 con 0x7f74b806ce80 2024-04-26T21:47:03.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.106+0000 7f74bda00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b815fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.107+0000 7f74b7e00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81638b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e156529a4f3c67ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.108+0000 7f74b7400640 1 -- 172.21.15.106:0/1510303513 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0031600 con 0x7f74b806ce80 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.108+0000 7f74b7400640 1 -- 172.21.15.106:0/1510303513 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a00317a0 con 0x7f74b806ce80 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.108+0000 7f74b7400640 1 -- 172.21.15.106:0/1510303513 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0031b70 con 0x7f74b806ce80 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.108+0000 7f74bfa00640 1 -- 172.21.15.106:0/1510303513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 msgr2=0x7f74b81638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.108+0000 7f74bfa00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81638b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 -- 172.21.15.106:0/1510303513 shutdown_connections 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b815fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 --2- 172.21.15.106:0/1510303513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81638b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 -- 172.21.15.106:0/1510303513 >> 172.21.15.106:0/1510303513 conn(0x7f74b80b2d50 msgr2=0x7f74b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 -- 172.21.15.106:0/1510303513 shutdown_connections 2024-04-26T21:47:03.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.109+0000 7f74bfa00640 1 -- 172.21.15.106:0/1510303513 wait complete. 2024-04-26T21:47:03.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.111+0000 7f74bfa00640 1 Processor -- start 2024-04-26T21:47:03.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.111+0000 7f74bfa00640 1 -- start start 2024-04-26T21:47:03.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b81598a0 0x7f74b809c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b81f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8163390 con 0x7f74b806ce80 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81630b0 con 0x7f74b81598a0 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f74b8160650 con 0x7f74b815f740 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59788/0 (socket says 172.21.15.106:59788) 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 -- 172.21.15.106:0/2140187221 learned_addr learned my addr 172.21.15.106:0/2140187221 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 -- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 msgr2=0x7f74b81f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b81f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 -- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b81598a0 msgr2=0x7f74b809c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b81598a0 0x7f74b809c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.112+0000 7f74bd000640 1 -- 172.21.15.106:0/2140187221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b80cf670 con 0x7f74b806ce80 2024-04-26T21:47:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74bd000640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c460 tx=0x7f74a800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8017070 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a8005ce0 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8007290 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b80cea60 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.113+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b80cf020 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.115+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b8069540 con 0x7f74b806ce80 2024-04-26T21:47:03.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.115+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74a801a070 con 0x7f74b806ce80 2024-04-26T21:47:03.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.117+0000 7f74b4c00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.117+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f74a808aee0 con 0x7f74b806ce80 2024-04-26T21:47:03.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.117+0000 7f74b7e00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.118+0000 7f74b7e00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 0x7f749406edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74a0034000 tx=0x7f74a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.122+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74a8059320 con 0x7f74b806ce80 2024-04-26T21:47:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.359+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f74b8073fc0 con 0x7f74b806ce80 2024-04-26T21:47:03.361 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-26T21:47:03.360+0000 7fca30c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:47:03.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.364+0000 7f74b4c00640 1 -- 172.21.15.106:0/2140187221 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f74a8058c20 con 0x7f74b806ce80 2024-04-26T21:47:03.365 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.366+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 msgr2=0x7f749406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.366+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 0x7f749406edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74a0034000 tx=0x7f74a002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.366+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 msgr2=0x7f74b81479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.366+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c460 tx=0x7f74a800c920 comp rx=0 tx=0).stop 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 shutdown_connections 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74b815f740 0x7f74b81f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b81598a0 0x7f74b809c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 --2- 172.21.15.106:0/2140187221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74b806ce80 0x7f74b81479e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 >> 172.21.15.106:0/2140187221 conn(0x7f74b80b2d50 msgr2=0x7f74b8161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 shutdown_connections 2024-04-26T21:47:03.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.367+0000 7f74bfa00640 1 -- 172.21.15.106:0/2140187221 wait complete. 2024-04-26T21:47:03.387 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:47:03.396 INFO:tasks.ceph.mds.d.smithi204.stderr: -4200> 2024-04-26T21:45:57.496+0000 7f1904000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:03.478 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-04-26T21:47:03.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.588+0000 7fb83ae00640 1 Processor -- start 2024-04-26T21:47:03.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.588+0000 7fb83ae00640 1 -- start start 2024-04-26T21:47:03.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb834147f20 0x7fb8340b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb83ae00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8340ade20 con 0x7fb83406ce80 2024-04-26T21:47:03.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb83ae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8340ae100 con 0x7fb834073da0 2024-04-26T21:47:03.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb83ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8340ae3e0 con 0x7fb834147f20 2024-04-26T21:47:03.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb838e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb83406ce80 0x7fb8340aeec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40356/0 (socket says 172.21.15.106:40356) 2024-04-26T21:47:03.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb838e00640 1 -- 172.21.15.106:0/1844003789 learned_addr learned my addr 172.21.15.106:0/1844003789 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:03.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.590+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb834147f20 0x7fb8340b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737988893 0 0) 0x7fb8340ade20 con 0x7fb83406ce80 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb818003520 con 0x7fb83406ce80 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3050926209 0 0) 0x7fb818003520 con 0x7fb83406ce80 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb834147f20 msgr2=0x7fb8340b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb834147f20 0x7fb8340b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 >> v1:172.21.15.204:6789/0 conn(0x7fb834073da0 legacy=0x7fb83406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8340ae6c0 con 0x7fb83406ce80 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fb82c001670 con 0x7fb83406ce80 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 >> v1:172.21.15.106:6789/0 conn(0x7fb83406ce80 legacy=0x7fb8340aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb818003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb818004390 0x7fb818004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb818005010 0x7fb81800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.591+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340ae3e0 con 0x7fb8180039e0 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb81800bf30 con 0x7fb818005010 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb81800c130 con 0x7fb818004390 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb834147f20 0x7fb8340b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb818003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb838e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb818005010 0x7fb81800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833400640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb818004390 0x7fb818004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 -- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb818004390 msgr2=0x7fb818004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb818004390 0x7fb818004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 -- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb818005010 msgr2=0x7fb81800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb818005010 0x7fb81800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.592+0000 7fb833e00640 1 -- 172.21.15.106:0/1844003789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb818003520 con 0x7fb8180039e0 2024-04-26T21:47:03.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.593+0000 7fb833e00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb818003dc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8340d1dc0 tx=0x7fb818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff710ba2afcb50a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824036070 con 0x7fb8180039e0 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824004030 con 0x7fb8180039e0 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb832a00640 1 -- 172.21.15.106:0/1844003789 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8240097d0 con 0x7fb8180039e0 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 -- 172.21.15.106:0/1844003789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 msgr2=0x7fb818003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1844003789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb818003dc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8340d1dc0 tx=0x7fb818005830 comp rx=0 tx=0).stop 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb838e00640 1 -- 172.21.15.106:0/1844003789 reap_dead start 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 -- 172.21.15.106:0/1844003789 shutdown_connections 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 -- 172.21.15.106:0/1844003789 >> 172.21.15.106:0/1844003789 conn(0x7fb8340b2d50 msgr2=0x7fb8340b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 -- 172.21.15.106:0/1844003789 shutdown_connections 2024-04-26T21:47:03.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.594+0000 7fb83ae00640 1 -- 172.21.15.106:0/1844003789 wait complete. 2024-04-26T21:47:03.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.596+0000 7fb83ae00640 1 Processor -- start 2024-04-26T21:47:03.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.596+0000 7fb83ae00640 1 -- start start 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8340ce170 0x7fb8340ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb834146170 0x7fb8341465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb83406e220 con 0x7fb8180039e0 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb834074200 con 0x7fb834146170 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.597+0000 7fb83ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8340b5c20 con 0x7fb8340ce170 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59806/0 (socket says 172.21.15.106:59806) 2024-04-26T21:47:03.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 -- 172.21.15.106:0/1299219931 learned_addr learned my addr 172.21.15.106:0/1299219931 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:03.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 -- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8340ce170 msgr2=0x7fb8340ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8340ce170 0x7fb8340ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 -- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb834146170 msgr2=0x7fb8341465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb834146170 0x7fb8341465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 -- 172.21.15.106:0/1299219931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83409d6a0 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.598+0000 7fb833e00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bf0 tx=0x7fb824004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.599+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824041020 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.599+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824003c40 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.599+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824036040 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.599+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83409d940 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.599+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83409df00 con 0x7fb8180039e0 2024-04-26T21:47:03.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.600+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb834069650 con 0x7fb8180039e0 2024-04-26T21:47:03.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.601+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb824044020 con 0x7fb8180039e0 2024-04-26T21:47:03.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.602+0000 7fb82be00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 0x7fb81406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.602+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fb8240ac160 con 0x7fb8180039e0 2024-04-26T21:47:03.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.602+0000 7fb833400640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 0x7fb81406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.603+0000 7fb833400640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 0x7fb81406ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.607+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb82404b070 con 0x7fb8180039e0 2024-04-26T21:47:03.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:03.848+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb83406cb10 con 0x7fb8180039e0 2024-04-26T21:47:04.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.367+0000 7fb82be00640 1 -- 172.21.15.106:0/1299219931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fb82407a420 con 0x7fb8180039e0 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 msgr2=0x7fb81406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 0x7fb81406ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).stop 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 msgr2=0x7fb83415ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bf0 tx=0x7fb824004730 comp rx=0 tx=0).stop 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 shutdown_connections 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb834146170 0x7fb8341465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb81406c920 0x7fb81406ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8340ce170 0x7fb8340ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 --2- 172.21.15.106:0/1299219931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8180039e0 0x7fb83415ea70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 >> 172.21.15.106:0/1299219931 conn(0x7fb8340b2d50 msgr2=0x7fb8340d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 shutdown_connections 2024-04-26T21:47:04.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.371+0000 7fb83ae00640 1 -- 172.21.15.106:0/1299219931 wait complete. 2024-04-26T21:47:04.393 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:47:04.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.601+0000 7f8dddc00640 1 Processor -- start 2024-04-26T21:47:04.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.601+0000 7f8dddc00640 1 -- start start 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dddc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8dd806e980 0x7f8dd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dddc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd806f7b0 con 0x7f8dd815b4a0 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dddc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd80b4540 con 0x7f8dd806f320 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dddc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd815b880 con 0x7f8dd806e980 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dd7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8dd815b4a0 0x7f8dd806fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40360/0 (socket says 172.21.15.106:40360) 2024-04-26T21:47:04.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dd7e00640 1 -- 172.21.15.106:0/744500120 learned_addr learned my addr 172.21.15.106:0/744500120 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:04.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.602+0000 7f8dd7400640 1 --2- 172.21.15.106:0/744500120 >> v2:172.21.15.106:3300/0 conn(0x7f8dd806e980 0x7f8dd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.603+0000 7f8dd7400640 1 -- 172.21.15.106:0/744500120 >> v1:172.21.15.204:6789/0 conn(0x7f8dd806f320 legacy=0x7f8dd8072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:04.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.603+0000 7f8dd7400640 1 -- 172.21.15.106:0/744500120 >> v1:172.21.15.106:6789/0 conn(0x7f8dd815b4a0 legacy=0x7f8dd806fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.603+0000 7f8dd7400640 1 -- 172.21.15.106:0/744500120 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80b4820 con 0x7f8dd806e980 2024-04-26T21:47:04.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.603+0000 7f8dd6a00640 1 -- 172.21.15.106:0/744500120 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200300721 0 0) 0x7f8dd806f7b0 con 0x7f8dd815b4a0 2024-04-26T21:47:04.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.604+0000 7f8dd7400640 1 --2- 172.21.15.106:0/744500120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd806e980 0x7f8dd806ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400a8c0 tx=0x7f8dc402b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c908410b55d4c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.604+0000 7f8dd6a00640 1 -- 172.21.15.106:0/744500120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4038070 con 0x7f8dd806e980 2024-04-26T21:47:04.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.604+0000 7f8dd6a00640 1 -- 172.21.15.106:0/744500120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc4033640 con 0x7f8dd806e980 2024-04-26T21:47:04.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.604+0000 7f8dd6a00640 1 -- 172.21.15.106:0/744500120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4033a10 con 0x7f8dd806e980 2024-04-26T21:47:04.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 -- 172.21.15.106:0/744500120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd806e980 msgr2=0x7f8dd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 --2- 172.21.15.106:0/744500120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd806e980 0x7f8dd806ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400a8c0 tx=0x7f8dc402b300 comp rx=0 tx=0).stop 2024-04-26T21:47:04.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 -- 172.21.15.106:0/744500120 shutdown_connections 2024-04-26T21:47:04.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 --2- 172.21.15.106:0/744500120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd806e980 0x7f8dd806ede0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 -- 172.21.15.106:0/744500120 >> 172.21.15.106:0/744500120 conn(0x7f8dd806c770 msgr2=0x7f8dd806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 -- 172.21.15.106:0/744500120 shutdown_connections 2024-04-26T21:47:04.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.605+0000 7f8dddc00640 1 -- 172.21.15.106:0/744500120 wait complete. 2024-04-26T21:47:04.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.607+0000 7f8dddc00640 1 Processor -- start 2024-04-26T21:47:04.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.607+0000 7f8dddc00640 1 -- start start 2024-04-26T21:47:04.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 0x7f8dd81469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd806f320 0x7f8dd80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd80b4de0 con 0x7f8dd815b4a0 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8071f00 con 0x7f8dd806e980 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8dddc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80d0f40 con 0x7f8dd806f320 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8ddd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8ddd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59828/0 (socket says 172.21.15.106:59828) 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.608+0000 7f8ddd200640 1 -- 172.21.15.106:0/1729709724 learned_addr learned my addr 172.21.15.106:0/1729709724 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dd7e00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 0x7f8dd81469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dd7400640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd806f320 0x7f8dd80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 -- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd806f320 msgr2=0x7f8dd80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd806f320 0x7f8dd80ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 -- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 msgr2=0x7f8dd81469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 0x7f8dd81469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 -- 172.21.15.106:0/1729709724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd8152030 con 0x7f8dd815b4a0 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dd7e00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 0x7f8dd81469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:04.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8ddd200640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c440 tx=0x7f8dd000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0011790 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd0017070 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0011b70 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.609+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd8152310 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.610+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd81528d0 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.611+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd80693c0 con 0x7f8dd815b4a0 2024-04-26T21:47:04.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.612+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dd0011d10 con 0x7f8dd815b4a0 2024-04-26T21:47:04.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.613+0000 7f8dcfe00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.613+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f8dd008b970 con 0x7f8dd815b4a0 2024-04-26T21:47:04.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.613+0000 7f8dd7e00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.614+0000 7f8dd7e00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8007fd0 tx=0x7f8dc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.617+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dd0059d60 con 0x7f8dd815b4a0 2024-04-26T21:47:04.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:04.817+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8dd8068a20 con 0x7f8dd815b4a0 2024-04-26T21:47:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.371+0000 7f8dcfe00640 1 -- 172.21.15.106:0/1729709724 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f8dd0059660 con 0x7f8dd815b4a0 2024-04-26T21:47:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 msgr2=0x7f8db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8007fd0 tx=0x7f8dc8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 msgr2=0x7f8dd80ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c440 tx=0x7f8dd000c900 comp rx=0 tx=0).stop 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 shutdown_connections 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dd815b4a0 0x7f8dd80ced60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd806f320 0x7f8dd80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 --2- 172.21.15.106:0/1729709724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dd806e980 0x7f8dd81469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 >> 172.21.15.106:0/1729709724 conn(0x7f8dd806c770 msgr2=0x7f8dd80cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.375+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 shutdown_connections 2024-04-26T21:47:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.376+0000 7f8dddc00640 1 -- 172.21.15.106:0/1729709724 wait complete. 2024-04-26T21:47:05.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:47:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.606+0000 7f8153e00640 1 Processor -- start 2024-04-26T21:47:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.606+0000 7f8153e00640 1 -- start start 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8153e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f814c15f6e0 0x7f814c15fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8153e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f814c0b3c30 0x7f814c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8153e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814c159af0 con 0x7f814c159710 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8153e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f814c072180 con 0x7f814c15f6e0 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8153e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f814c160210 con 0x7f814c0b3c30 2024-04-26T21:47:05.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8150a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f814c15f6e0 0x7f814c15fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8150a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f814c15f6e0 0x7f814c15fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59836/0 (socket says 172.21.15.106:59836) 2024-04-26T21:47:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8150a00640 1 -- 172.21.15.106:0/1895431739 learned_addr learned my addr 172.21.15.106:0/1895431739 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.608+0000 7f8151e00640 1 --2- 172.21.15.106:0/1895431739 >> v2:172.21.15.204:3300/0 conn(0x7f814c0b3c30 0x7f814c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.609+0000 7f8150a00640 1 -- 172.21.15.106:0/1895431739 >> v2:172.21.15.204:3300/0 conn(0x7f814c0b3c30 msgr2=0x7f814c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.609+0000 7f8150a00640 1 --2- 172.21.15.106:0/1895431739 >> v2:172.21.15.204:3300/0 conn(0x7f814c0b3c30 0x7f814c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.609+0000 7f8150a00640 1 -- 172.21.15.106:0/1895431739 >> v1:172.21.15.106:6789/0 conn(0x7f814c159710 legacy=0x7f814c072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.609+0000 7f8150a00640 1 -- 172.21.15.106:0/1895431739 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c1604f0 con 0x7f814c15f6e0 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.609+0000 7f814be00640 1 -- 172.21.15.106:0/1895431739 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486100663 0 0) 0x7f814c159af0 con 0x7f814c159710 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.610+0000 7f8150a00640 1 --2- 172.21.15.106:0/1895431739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c15f6e0 0x7f814c15fb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f814c06efe0 tx=0x7f813802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d5448f1bbd542f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.610+0000 7f814be00640 1 -- 172.21.15.106:0/1895431739 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138037070 con 0x7f814c15f6e0 2024-04-26T21:47:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.610+0000 7f814be00640 1 -- 172.21.15.106:0/1895431739 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8138032440 con 0x7f814c15f6e0 2024-04-26T21:47:05.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.610+0000 7f814be00640 1 -- 172.21.15.106:0/1895431739 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138032810 con 0x7f814c15f6e0 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 -- 172.21.15.106:0/1895431739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c15f6e0 msgr2=0x7f814c15fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 --2- 172.21.15.106:0/1895431739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c15f6e0 0x7f814c15fb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f814c06efe0 tx=0x7f813802a270 comp rx=0 tx=0).stop 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 -- 172.21.15.106:0/1895431739 shutdown_connections 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 --2- 172.21.15.106:0/1895431739 >> v2:172.21.15.204:3300/0 conn(0x7f814c0b3c30 0x7f814c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 --2- 172.21.15.106:0/1895431739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c15f6e0 0x7f814c15fb90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 -- 172.21.15.106:0/1895431739 >> 172.21.15.106:0/1895431739 conn(0x7f814c0d0c40 msgr2=0x7f814c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 -- 172.21.15.106:0/1895431739 shutdown_connections 2024-04-26T21:47:05.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.611+0000 7f8153e00640 1 -- 172.21.15.106:0/1895431739 wait complete. 2024-04-26T21:47:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.613+0000 7f8153e00640 1 Processor -- start 2024-04-26T21:47:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.613+0000 7f8153e00640 1 -- start start 2024-04-26T21:47:05.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f814c159710 0x7f814c09c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f814c15f6e0 0x7f814c09dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f814c070130 con 0x7f814c0b3c30 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f814c072720 con 0x7f814c159710 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8153e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f814c073350 con 0x7f814c15f6e0 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8151400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8151400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59846/0 (socket says 172.21.15.106:59846) 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.614+0000 7f8151400640 1 -- 172.21.15.106:0/2769682485 learned_addr learned my addr 172.21.15.106:0/2769682485 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 -- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f814c15f6e0 msgr2=0x7f814c09dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f814c15f6e0 0x7f814c09dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8150a00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f814c159710 0x7f814c09c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f814c15f6e0 0x7f814c09dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 -- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f814c159710 msgr2=0x7f814c09c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f814c159710 0x7f814c09c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 -- 172.21.15.106:0/2769682485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c0b03e0 con 0x7f814c0b3c30 2024-04-26T21:47:05.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8151400640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f814000c410 tx=0x7f814000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140017070 con 0x7f814c0b3c30 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81400035f0 con 0x7f814c0b3c30 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400039e0 con 0x7f814c0b3c30 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c09bdf0 con 0x7f814c0b3c30 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.615+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c0b0930 con 0x7f814c0b3c30 2024-04-26T21:47:05.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.617+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c069650 con 0x7f814c0b3c30 2024-04-26T21:47:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.618+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f814001a070 con 0x7f814c0b3c30 2024-04-26T21:47:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.619+0000 7f8149600640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 0x7f812c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.619+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f814008aff0 con 0x7f814c0b3c30 2024-04-26T21:47:05.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.619+0000 7f8150a00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 0x7f812c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.620+0000 7f8150a00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 0x7f812c06ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8138034000 tx=0x7f813802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.624+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81400593d0 con 0x7f814c0b3c30 2024-04-26T21:47:05.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:05.824+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f814c06e8d0 con 0x7f814c0b3c30 2024-04-26T21:47:06.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.375+0000 7f8149600640 1 -- 172.21.15.106:0/2769682485 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f8140058cd0 con 0x7f814c0b3c30 2024-04-26T21:47:06.376 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-26T21:47:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 msgr2=0x7f812c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 0x7f812c06ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8138034000 tx=0x7f813802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 msgr2=0x7f814c148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f814000c410 tx=0x7f814000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 shutdown_connections 2024-04-26T21:47:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f812c06c920 0x7f812c06ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f814c15f6e0 0x7f814c09dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f814c159710 0x7f814c09c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 --2- 172.21.15.106:0/2769682485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c0b3c30 0x7f814c148740 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.380+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 >> 172.21.15.106:0/2769682485 conn(0x7f814c0d0c40 msgr2=0x7f814c1611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.383+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 shutdown_connections 2024-04-26T21:47:06.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.383+0000 7f8153e00640 1 -- 172.21.15.106:0/2769682485 wait complete. 2024-04-26T21:47:06.413 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:47:06.413 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:47:06.414 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:47:06.414 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:06.414 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:47:06.414 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:47:06.414 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:47:06.414 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:06.414 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:47:06.415 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:47:06.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:47:06.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:06.415 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:47:06.415 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:47:06.415 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:47:06.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:06.430 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:47:06.430 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:47:06.434 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:47:06.434 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:47:06.435 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:47:06.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:47:06.437 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:47:06.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:47:06.480 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:47:06.481 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:06.481 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:47:06.482 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:06.485 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:47:06.485 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:06.486 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:47:06.486 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:06.487 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:47:06.638 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-26T21:47:06.639 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-04-26T21:47:06.640 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-26T21:47:06.640 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-26T21:47:06.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.675+0000 7f6447a00640 1 Processor -- start 2024-04-26T21:47:06.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.675+0000 7f6447a00640 1 -- start start 2024-04-26T21:47:06.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6447a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f644009ba50 0x7f644009be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6447a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f644006f230 0x7f644006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6447a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644009c370 con 0x7f64400b58a0 2024-04-26T21:47:06.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6447a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6440070420 con 0x7f644006f230 2024-04-26T21:47:06.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6447a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f644006e790 con 0x7f644009ba50 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f643fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f64400b58a0 0x7f644006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40386/0 (socket says 172.21.15.106:40386) 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f643fe00640 1 -- 172.21.15.106:0/1577974087 learned_addr learned my addr 172.21.15.106:0/1577974087 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6445000640 1 --2- 172.21.15.106:0/1577974087 >> v2:172.21.15.204:3300/0 conn(0x7f644009ba50 0x7f644009be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.676+0000 7f6445a00640 1 --2- 172.21.15.106:0/1577974087 >> v2:172.21.15.106:3300/0 conn(0x7f644006f230 0x7f644006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.677+0000 7f6445000640 1 -- 172.21.15.106:0/1577974087 >> v2:172.21.15.106:3300/0 conn(0x7f644006f230 msgr2=0x7f644006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.677+0000 7f6445000640 1 --2- 172.21.15.106:0/1577974087 >> v2:172.21.15.106:3300/0 conn(0x7f644006f230 0x7f644006db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.677+0000 7f6445000640 1 -- 172.21.15.106:0/1577974087 >> v1:172.21.15.106:6789/0 conn(0x7f64400b58a0 legacy=0x7f644006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.677+0000 7f6445000640 1 -- 172.21.15.106:0/1577974087 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6440070670 con 0x7f644009ba50 2024-04-26T21:47:06.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.678+0000 7f6445000640 1 --2- 172.21.15.106:0/1577974087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644009ba50 0x7f644009be30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64400767a0 tx=0x7f643402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ed943ddd89a2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.678+0000 7f643f400640 1 -- 172.21.15.106:0/1577974087 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434037070 con 0x7f644009ba50 2024-04-26T21:47:06.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.678+0000 7f643f400640 1 -- 172.21.15.106:0/1577974087 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434032440 con 0x7f644009ba50 2024-04-26T21:47:06.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.678+0000 7f643f400640 1 -- 172.21.15.106:0/1577974087 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643402ae60 con 0x7f644009ba50 2024-04-26T21:47:06.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 -- 172.21.15.106:0/1577974087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644009ba50 msgr2=0x7f644009be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 --2- 172.21.15.106:0/1577974087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644009ba50 0x7f644009be30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64400767a0 tx=0x7f643402a110 comp rx=0 tx=0).stop 2024-04-26T21:47:06.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 -- 172.21.15.106:0/1577974087 shutdown_connections 2024-04-26T21:47:06.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 --2- 172.21.15.106:0/1577974087 >> v2:172.21.15.106:3300/0 conn(0x7f644006f230 0x7f644006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 --2- 172.21.15.106:0/1577974087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644009ba50 0x7f644009be30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 -- 172.21.15.106:0/1577974087 >> 172.21.15.106:0/1577974087 conn(0x7f64400b4b70 msgr2=0x7f64400b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 -- 172.21.15.106:0/1577974087 shutdown_connections 2024-04-26T21:47:06.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.679+0000 7f6447a00640 1 -- 172.21.15.106:0/1577974087 wait complete. 2024-04-26T21:47:06.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.681+0000 7f6447a00640 1 Processor -- start 2024-04-26T21:47:06.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.681+0000 7f6447a00640 1 -- start start 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644006f230 0x7f64401d5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64400b58a0 0x7f64400cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f644006ecc0 con 0x7f644009ba50 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f644006e9e0 con 0x7f644006f230 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f6447a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64400cf440 con 0x7f64400b58a0 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f643fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f643fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59908/0 (socket says 172.21.15.106:59908) 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.682+0000 7f643fe00640 1 -- 172.21.15.106:0/2434840878 learned_addr learned my addr 172.21.15.106:0/2434840878 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f6445000640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644006f230 0x7f64401d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f6445a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64400b58a0 0x7f64400cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 -- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64400b58a0 msgr2=0x7f64400cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64400b58a0 0x7f64400cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 -- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644006f230 msgr2=0x7f64401d5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644006f230 0x7f64401d5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 -- 172.21.15.106:0/2434840878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64400b8f50 con 0x7f644009ba50 2024-04-26T21:47:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643fe00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c400 tx=0x7f642c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.683+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c017070 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.684+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c005ce0 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.684+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0072f0 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.684+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64400b79b0 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.684+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64400b7f30 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.685+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440069540 con 0x7f644009ba50 2024-04-26T21:47:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.686+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f642c01a070 con 0x7f644009ba50 2024-04-26T21:47:06.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.686+0000 7f643cc00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 0x7f642006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.687+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f642c08ab80 con 0x7f644009ba50 2024-04-26T21:47:06.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.687+0000 7f6445000640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 0x7f642006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.687+0000 7f6445000640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 0x7f642006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f643402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.691+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f642c058c60 con 0x7f644009ba50 2024-04-26T21:47:06.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:06.908+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f64400d19a0 con 0x7f644009ba50 2024-04-26T21:47:07.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.410+0000 7f643cc00640 1 -- 172.21.15.106:0/2434840878 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f642c05cf60 con 0x7f644009ba50 2024-04-26T21:47:07.412 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-26T21:47:07.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.414+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 msgr2=0x7f642006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.414+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 0x7f642006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f643402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:07.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.414+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 msgr2=0x7f64400b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.414+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c400 tx=0x7f642c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 shutdown_connections 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f642006c8f0 0x7f642006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64400b58a0 0x7f64400cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f644009ba50 0x7f64400b8a10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 --2- 172.21.15.106:0/2434840878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644006f230 0x7f64401d5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 >> 172.21.15.106:0/2434840878 conn(0x7f64400b4b70 msgr2=0x7f64400aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 shutdown_connections 2024-04-26T21:47:07.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.415+0000 7f6447a00640 1 -- 172.21.15.106:0/2434840878 wait complete. 2024-04-26T21:47:07.439 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:47:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.631+0000 7fe752e00640 1 Processor -- start 2024-04-26T21:47:07.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.632+0000 7fe752e00640 1 -- start start 2024-04-26T21:47:07.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.633+0000 7fe752e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe74c15cdf0 0x7fe74c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.633+0000 7fe752e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe74c0cd650 0x7fe74c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.633+0000 7fe752e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe74c0af9a0 con 0x7fe74c0b4f50 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.633+0000 7fe752e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe74c0ae2c0 con 0x7fe74c15cdf0 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.633+0000 7fe752e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe74c0b05b0 con 0x7fe74c0cd650 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe74c15cdf0 0x7fe74c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe74c15cdf0 0x7fe74c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59914/0 (socket says 172.21.15.106:59914) 2024-04-26T21:47:07.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 -- 172.21.15.106:0/525656340 learned_addr learned my addr 172.21.15.106:0/525656340 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe743400640 1 --2- 172.21.15.106:0/525656340 >> v2:172.21.15.204:3300/0 conn(0x7fe74c0cd650 0x7fe74c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 -- 172.21.15.106:0/525656340 >> v2:172.21.15.204:3300/0 conn(0x7fe74c0cd650 msgr2=0x7fe74c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 --2- 172.21.15.106:0/525656340 >> v2:172.21.15.204:3300/0 conn(0x7fe74c0cd650 0x7fe74c06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 -- 172.21.15.106:0/525656340 >> v1:172.21.15.204:6789/0 conn(0x7fe74c0b4f50 legacy=0x7fe74c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.634+0000 7fe74be00640 1 -- 172.21.15.106:0/525656340 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c0afc80 con 0x7fe74c15cdf0 2024-04-26T21:47:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.636+0000 7fe74be00640 1 --2- 172.21.15.106:0/525656340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c0add80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0d1d00 tx=0x7fe73c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453bd6e9a246e99c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.636+0000 7fe74b400640 1 -- 172.21.15.106:0/525656340 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c037070 con 0x7fe74c15cdf0 2024-04-26T21:47:07.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.636+0000 7fe74b400640 1 -- 172.21.15.106:0/525656340 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c032440 con 0x7fe74c15cdf0 2024-04-26T21:47:07.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.636+0000 7fe74b400640 1 -- 172.21.15.106:0/525656340 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c032810 con 0x7fe74c15cdf0 2024-04-26T21:47:07.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 -- 172.21.15.106:0/525656340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 msgr2=0x7fe74c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 --2- 172.21.15.106:0/525656340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c0add80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0d1d00 tx=0x7fe73c02a230 comp rx=0 tx=0).stop 2024-04-26T21:47:07.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 -- 172.21.15.106:0/525656340 shutdown_connections 2024-04-26T21:47:07.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 --2- 172.21.15.106:0/525656340 >> v2:172.21.15.204:3300/0 conn(0x7fe74c0cd650 0x7fe74c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 --2- 172.21.15.106:0/525656340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c0add80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 -- 172.21.15.106:0/525656340 >> 172.21.15.106:0/525656340 conn(0x7fe74c0b2d50 msgr2=0x7fe74c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 -- 172.21.15.106:0/525656340 shutdown_connections 2024-04-26T21:47:07.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.637+0000 7fe752e00640 1 -- 172.21.15.106:0/525656340 wait complete. 2024-04-26T21:47:07.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.639+0000 7fe752e00640 1 Processor -- start 2024-04-26T21:47:07.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.639+0000 7fe752e00640 1 -- start start 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0b4f50 0x7fe74c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c0cd650 0x7fe74c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c15d1d0 con 0x7fe74c15cdf0 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0b0240 con 0x7fe74c0cd650 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe752e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe74c15d350 con 0x7fe74c0b4f50 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe750e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe750e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59926/0 (socket says 172.21.15.106:59926) 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.640+0000 7fe750e00640 1 -- 172.21.15.106:0/1226400901 learned_addr learned my addr 172.21.15.106:0/1226400901 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:07.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe74be00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0b4f50 0x7fe74c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe743400640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c0cd650 0x7fe74c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 -- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0b4f50 msgr2=0x7fe74c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0b4f50 0x7fe74c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 -- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c0cd650 msgr2=0x7fe74c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c0cd650 0x7fe74c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 -- 172.21.15.106:0/1226400901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c1f2970 con 0x7fe74c15cdf0 2024-04-26T21:47:07.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe750e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe74400c7b0 tx=0x7fe74400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.641+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744004b60 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.642+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe744004d00 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.642+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400aa90 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.642+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c1f2c50 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.642+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1f3210 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.643+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c069540 con 0x7fe74c15cdf0 2024-04-26T21:47:07.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.644+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe74400ac30 con 0x7fe74c15cdf0 2024-04-26T21:47:07.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.645+0000 7fe748c00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.645+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fe74408bb00 con 0x7fe74c15cdf0 2024-04-26T21:47:07.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.646+0000 7fe74be00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.646+0000 7fe74be00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 0x7fe72806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe73c034000 tx=0x7fe73c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.650+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe744020070 con 0x7fe74c15cdf0 2024-04-26T21:47:07.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.923+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe74c076760 con 0x7fe74c15cdf0 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.923+0000 7fe748c00640 1 -- 172.21.15.106:0/1226400901 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) ==== 80+0+6645 (secure 0 0 0) 0x7fe74408be20 con 0x7fe74c15cdf0 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmnY2XFBAAtaQO3LQwin4e9iNrklJH6w==", 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.925 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-26T21:47:07.926 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxm684cMxAAYTf8acqdf2CCeHokqOS0Xg==", 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.927 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmLYa9IBAAndFvm1tzi44OqYb6ttR3Sw==", 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.928 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxm90qUBhAA9teVVNbkqfjkCWMBcZK//Q==", 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:07.929 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBVICxmvpP+ARAAApDQrRCTWu21gf/CGyROOg==", 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.930 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBYICxmMM6lCxAANvEZrGQcAR75+Xy8ktUM5A==", 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.931 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.932 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBbICxmZPvaFBAA/2/DymuSvYvc1WYgY1jO/Q==", 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.933 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBeICxmhYIOIBAAa0ISn+YDALtH4Ma087UI6Q==", 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.934 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBkICxmTlgVNhAA3FMFqR8eN9aQooYqEN2kxQ==", 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.935 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBoICxmoSRBBhAAVLMhNepkCpcdbgJWdcr8FA==", 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.936 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBrICxm7HdTEBAA5myIxNTNd3FaNjEbMiILEQ==", 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.937 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBuICxmrWiyKRAAxRDKvFrBA1YRdRif8ZH/ow==", 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.938 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q==", 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.939 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmjz8uJRAA7zXqZtFgUCBE9tb+7barEw==", 2024-04-26T21:47:07.940 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-26T21:47:07.941 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmCucaGxAAbxoXORUIaoRjav5XMYwKZA==", 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.942 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmsp5GLhAAEuL2LpnxmDBCfgz5b3AcuQ==", 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.943 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBPICxmLtcjFBAADhgvakKCSkF7zmm7uRGekA==", 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.944 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mds", 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxmdLjmABAA4QCE+Bya8Rx+as9bd3cW+A==", 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.945 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T21:47:07.946 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.946 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.946 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.946 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T21:47:07.946 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxmXNXmABAAVqWi/hQY+9uIIlN1v/7kPw==", 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.947 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-osd", 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxm4e3mABAAHe+hBQ6IN/eH8wlnjHcvNg==", 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.948 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxmdQrnABAABow/pGQ5dY8tzVKH766Mbw==", 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.949 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.950 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.950 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T21:47:07.950 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxmjCjnABAARl6JTwkqyG2eknOoNlpnKA==", 2024-04-26T21:47:07.950 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.950 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQB8ICxmUEHnABAAW2BGWAnrcyCFR+Hetfk0TA==", 2024-04-26T21:47:07.951 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.952 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T21:47:07.952 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.952 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.952 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.952 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmzUzsDBAAIx87eHODdurs4RGojAb/nQ==", 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.953 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.954 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:07.954 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:07.954 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-26T21:47:07.954 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmyV5pAhAApZSAbzZOWb572E7vzC6aJQ==", 2024-04-26T21:47:07.954 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:07.955 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-26T21:47:07.956 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:47:07.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 msgr2=0x7fe72806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 0x7fe72806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe73c034000 tx=0x7fe73c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:07.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 msgr2=0x7fe74c158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe74400c7b0 tx=0x7fe74400cc70 comp rx=0 tx=0).stop 2024-04-26T21:47:07.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 shutdown_connections 2024-04-26T21:47:07.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74c15cdf0 0x7fe74c158430 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c0cd650 0x7fe74c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 --2- 172.21.15.106:0/1226400901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0b4f50 0x7fe74c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 >> 172.21.15.106:0/1226400901 conn(0x7fe74c0b2d50 msgr2=0x7fe74c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 shutdown_connections 2024-04-26T21:47:07.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:07.926+0000 7fe752e00640 1 -- 172.21.15.106:0/1226400901 wait complete. 2024-04-26T21:47:07.962 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T21:47:08.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.164+0000 7f67a1200640 1 Processor -- start 2024-04-26T21:47:08.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.164+0000 7f67a1200640 1 -- start start 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f67a1200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 0x7f679c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f67a1200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f679c0726a0 0x7f679c0d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f67a1200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679c0afe30 con 0x7f679c0af2e0 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f67a1200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f679c0afc00 con 0x7f679c0726a0 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f67a1200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0b0110 con 0x7f679c072260 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f679b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f679c0af2e0 0x7f679c073bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38116/0 (socket says 172.21.15.106:38116) 2024-04-26T21:47:08.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f679b400640 1 -- 172.21.15.106:0/1922977043 learned_addr learned my addr 172.21.15.106:0/1922977043 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f679aa00640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 0x7f679c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.166+0000 7f679be00640 1 --2- 172.21.15.106:0/1922977043 >> v2:172.21.15.204:3300/0 conn(0x7f679c0726a0 0x7f679c0d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3502154255 0 0) 0x7f679c0afe30 con 0x7f679c0af2e0 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6780003520 con 0x7f679c0af2e0 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095707428 0 0) 0x7f6780003520 con 0x7f679c0af2e0 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 msgr2=0x7f679c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 0x7f679c0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 >> v2:172.21.15.204:3300/0 conn(0x7f679c0726a0 msgr2=0x7f679c0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 --2- 172.21.15.106:0/1922977043 >> v2:172.21.15.204:3300/0 conn(0x7f679c0726a0 0x7f679c0d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c0b03f0 con 0x7f679c0af2e0 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f678c001670 con 0x7f679c0af2e0 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 >> v1:172.21.15.106:6789/0 conn(0x7f679c0af2e0 legacy=0x7f679c073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67800039b0 0x7f6780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6780004800 0x7f6780004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 0x7f67800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0b0110 con 0x7f6780004800 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f678000b9c0 con 0x7f67800039b0 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f678000bb40 con 0x7f679c072260 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.167+0000 7f679aa00640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6780004800 0x7f6780004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.168+0000 7f679aa00640 1 -- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 msgr2=0x7f67800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.168+0000 7f679aa00640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c072260 0x7f67800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.168+0000 7f679aa00640 1 -- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67800039b0 msgr2=0x7f6780003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.168+0000 7f679aa00640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67800039b0 0x7f6780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.168+0000 7f679aa00640 1 -- 172.21.15.106:0/1922977043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6780003520 con 0x7f6780004800 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.169+0000 7f679aa00640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6780004800 0x7f6780004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6788009ac0 tx=0x7f6780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bd775750d8acfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.169+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788036070 con 0x7f6780004800 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.169+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6788031400 con 0x7f6780004800 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.169+0000 7f679a000640 1 -- 172.21.15.106:0/1922977043 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788009690 con 0x7f6780004800 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 -- 172.21.15.106:0/1922977043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6780004800 msgr2=0x7f6780004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 --2- 172.21.15.106:0/1922977043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6780004800 0x7f6780004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6788009ac0 tx=0x7f6780005830 comp rx=0 tx=0).stop 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f679be00640 1 -- 172.21.15.106:0/1922977043 reap_dead start 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 -- 172.21.15.106:0/1922977043 shutdown_connections 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 -- 172.21.15.106:0/1922977043 >> 172.21.15.106:0/1922977043 conn(0x7f679c06c560 msgr2=0x7f679c06c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 -- 172.21.15.106:0/1922977043 shutdown_connections 2024-04-26T21:47:08.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.170+0000 7f67a1200640 1 -- 172.21.15.106:0/1922977043 wait complete. 2024-04-26T21:47:08.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.172+0000 7f67a1200640 1 Processor -- start 2024-04-26T21:47:08.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.172+0000 7f67a1200640 1 -- start start 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 0x7f679c161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c1507d0 0x7f679c152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0b5dd0 con 0x7f679c15ff10 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0b5c50 con 0x7f6780004800 2024-04-26T21:47:08.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f67a1200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0af6c0 con 0x7f679c1507d0 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38074/0 (socket says 172.21.15.106:38074) 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 -- 172.21.15.106:0/1308952959 learned_addr learned my addr 172.21.15.106:0/1308952959 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679b400640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 0x7f679c161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679be00640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c1507d0 0x7f679c152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 -- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c1507d0 msgr2=0x7f679c152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c1507d0 0x7f679c152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.173+0000 7f679aa00640 1 -- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 msgr2=0x7f679c161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f679aa00640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 0x7f679c161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f679aa00640 1 -- 172.21.15.106:0/1308952959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c1606d0 con 0x7f679c15ff10 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f679b400640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 0x7f679c161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f679aa00640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6788009be0 tx=0x7f6788004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788040020 con 0x7f679c15ff10 2024-04-26T21:47:08.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67880038d0 con 0x7f679c15ff10 2024-04-26T21:47:08.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c1609b0 con 0x7f679c15ff10 2024-04-26T21:47:08.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788036040 con 0x7f679c15ff10 2024-04-26T21:47:08.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.174+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c09d620 con 0x7f679c15ff10 2024-04-26T21:47:08.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.175+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c0693c0 con 0x7f679c15ff10 2024-04-26T21:47:08.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.176+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6788043020 con 0x7f679c15ff10 2024-04-26T21:47:08.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.177+0000 7f6793400640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 0x7f677c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.178+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f67880ab890 con 0x7f679c15ff10 2024-04-26T21:47:08.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.178+0000 7f679b400640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 0x7f677c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.178+0000 7f679b400640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 0x7f677c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f678c001eb0 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.182+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f678804a070 con 0x7f679c15ff10 2024-04-26T21:47:08.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.453+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f679c06e290 con 0x7f679c15ff10 2024-04-26T21:47:08.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.457+0000 7f6793400640 1 -- 172.21.15.106:0/1308952959 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f678807a000 con 0x7f679c15ff10 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.459+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 msgr2=0x7f677c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.459+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 0x7f677c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f678c001eb0 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.459+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 msgr2=0x7f679c0b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.459+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6788009be0 tx=0x7f6788004730 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 shutdown_connections 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677c06c920 0x7f677c06ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f679c1507d0 0x7f679c152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f679c15ff10 0x7f679c0b1620 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 --2- 172.21.15.106:0/1308952959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6780004800 0x7f679c161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 >> 172.21.15.106:0/1308952959 conn(0x7f679c06c560 msgr2=0x7f679c073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 shutdown_connections 2024-04-26T21:47:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.460+0000 7f67a1200640 1 -- 172.21.15.106:0/1308952959 wait complete. 2024-04-26T21:47:08.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T21:47:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.661+0000 7fdeee800640 1 Processor -- start 2024-04-26T21:47:08.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.662+0000 7fdeee800640 1 -- start start 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdee80cf640 0x7fdee80b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80af670 0x7fdee815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 0x7fdee80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdee80afb50 con 0x7fdee80cf640 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdee80afe30 con 0x7fdee815d720 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeee800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80b0110 con 0x7fdee80af670 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeed400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdee80cf640 0x7fdee80b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeed400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdee80cf640 0x7fdee80b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38084/0 (socket says 172.21.15.106:38084) 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.663+0000 7fdeed400640 1 -- 172.21.15.106:0/2890579365 learned_addr learned my addr 172.21.15.106:0/2890579365 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeede00640 1 --2- 172.21.15.106:0/2890579365 >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 0x7fdee80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeeca00640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80af670 0x7fdee815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeed400640 1 -- 172.21.15.106:0/2890579365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80af670 msgr2=0x7fdee815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeed400640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80af670 0x7fdee815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeed400640 1 -- 172.21.15.106:0/2890579365 >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 msgr2=0x7fdee80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeed400640 1 --2- 172.21.15.106:0/2890579365 >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 0x7fdee80d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeed400640 1 -- 172.21.15.106:0/2890579365 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80b03f0 con 0x7fdee80cf640 2024-04-26T21:47:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.664+0000 7fdeede00640 1 --2- 172.21.15.106:0/2890579365 >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 0x7fdee80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:08.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.665+0000 7fdeed400640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee80cf640 0x7fdee80b5920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdee80b5210 tx=0x7fdedc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f52f15edfaa945 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.665+0000 7fdee7e00640 1 -- 172.21.15.106:0/2890579365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc037070 con 0x7fdee80cf640 2024-04-26T21:47:08.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.665+0000 7fdee7e00640 1 -- 172.21.15.106:0/2890579365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc032440 con 0x7fdee80cf640 2024-04-26T21:47:08.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.665+0000 7fdee7e00640 1 -- 172.21.15.106:0/2890579365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc032810 con 0x7fdee80cf640 2024-04-26T21:47:08.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 -- 172.21.15.106:0/2890579365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee80cf640 msgr2=0x7fdee80b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee80cf640 0x7fdee80b5920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdee80b5210 tx=0x7fdedc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 -- 172.21.15.106:0/2890579365 shutdown_connections 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 --2- 172.21.15.106:0/2890579365 >> v2:172.21.15.204:3300/0 conn(0x7fdee815d720 0x7fdee80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80af670 0x7fdee815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 --2- 172.21.15.106:0/2890579365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee80cf640 0x7fdee80b5920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 -- 172.21.15.106:0/2890579365 >> 172.21.15.106:0/2890579365 conn(0x7fdee806c560 msgr2=0x7fdee806c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 -- 172.21.15.106:0/2890579365 shutdown_connections 2024-04-26T21:47:08.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.666+0000 7fdeee800640 1 -- 172.21.15.106:0/2890579365 wait complete. 2024-04-26T21:47:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.668+0000 7fdeee800640 1 Processor -- start 2024-04-26T21:47:08.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.668+0000 7fdeee800640 1 -- start start 2024-04-26T21:47:08.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 0x7fdee81569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80cf640 0x7fdee81f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdee80d12d0 con 0x7fdee815d720 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee80b6070 con 0x7fdee80af670 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeee800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80d1450 con 0x7fdee80cf640 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeede00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeede00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38088/0 (socket says 172.21.15.106:38088) 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.669+0000 7fdeede00640 1 -- 172.21.15.106:0/2430741682 learned_addr learned my addr 172.21.15.106:0/2430741682 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeed400640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 0x7fdee81569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeeca00640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80cf640 0x7fdee81f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 -- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80cf640 msgr2=0x7fdee81f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80cf640 0x7fdee81f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 -- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 msgr2=0x7fdee81569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 0x7fdee81569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 -- 172.21.15.106:0/2430741682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee809e890 con 0x7fdee815d720 2024-04-26T21:47:08.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeed400640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 0x7fdee81569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:08.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeede00640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009fa0 tx=0x7fdee000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee001a070 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee0003bb0 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004030 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.670+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee809baf0 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.671+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee809c0b0 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.672+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee80693c0 con 0x7fdee815d720 2024-04-26T21:47:08.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.673+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdee001d070 con 0x7fdee815d720 2024-04-26T21:47:08.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.673+0000 7fdee5600640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:08.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.674+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdee008ba10 con 0x7fdee815d720 2024-04-26T21:47:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.674+0000 7fdeed400640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.675+0000 7fdeed400640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 0x7fdec806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034000 tx=0x7fdedc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:08.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.678+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdee0010070 con 0x7fdee815d720 2024-04-26T21:47:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.931+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fdee8156ee0 con 0x7fdee815d720 2024-04-26T21:47:08.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.934+0000 7fdee5600640 1 -- 172.21.15.106:0/2430741682 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fdee005a200 con 0x7fdee815d720 2024-04-26T21:47:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 msgr2=0x7fdec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 0x7fdec806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034000 tx=0x7fdedc02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 msgr2=0x7fdee80b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009fa0 tx=0x7fdee000d560 comp rx=0 tx=0).stop 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 shutdown_connections 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdee815d720 0x7fdee80b27e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee80cf640 0x7fdee81f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 --2- 172.21.15.106:0/2430741682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee80af670 0x7fdee81569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 >> 172.21.15.106:0/2430741682 conn(0x7fdee806c560 msgr2=0x7fdee8073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 shutdown_connections 2024-04-26T21:47:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:08.937+0000 7fdeee800640 1 -- 172.21.15.106:0/2430741682 wait complete. 2024-04-26T21:47:08.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T21:47:09.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.154+0000 7f2d63e00640 1 Processor -- start 2024-04-26T21:47:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.155+0000 7f2d63e00640 1 -- start start 2024-04-26T21:47:09.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.156+0000 7f2d63e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d6415f270 0x7f2d640d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.156+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 0x7f2d640b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.156+0000 7f2d63e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d640d0ed0 con 0x7f2d640d3c90 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.156+0000 7f2d63e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2d640d2540 con 0x7f2d6415f270 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.156+0000 7f2d63e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d640cfe30 con 0x7f2d640d1df0 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d6415f270 0x7f2d640d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2d640d3c90 0x7f2d640d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38122/0 (socket says 172.21.15.106:38122) 2024-04-26T21:47:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62000640 1 -- 172.21.15.106:0/3692243645 learned_addr learned my addr 172.21.15.106:0/3692243645 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d63400640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 0x7f2d640b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62a00640 1 -- 172.21.15.106:0/3692243645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 msgr2=0x7f2d640b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d61600640 1 -- 172.21.15.106:0/3692243645 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1224944743 0 0) 0x7f2d54001490 con 0x7f2d640d3c90 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62a00640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 0x7f2d640b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62a00640 1 -- 172.21.15.106:0/3692243645 >> v1:172.21.15.106:6789/0 conn(0x7f2d640d3c90 legacy=0x7f2d640d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d62a00640 1 -- 172.21.15.106:0/3692243645 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d640d0110 con 0x7f2d6415f270 2024-04-26T21:47:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.157+0000 7f2d63400640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 0x7f2d640b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.158+0000 7f2d62a00640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d6415f270 0x7f2d640d3750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c09b1e650edf3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d61600640 1 -- 172.21.15.106:0/3692243645 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037070 con 0x7f2d6415f270 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d61600640 1 -- 172.21.15.106:0/3692243645 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c032440 con 0x7f2d6415f270 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d61600640 1 -- 172.21.15.106:0/3692243645 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c032810 con 0x7f2d6415f270 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d63e00640 1 -- 172.21.15.106:0/3692243645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d6415f270 msgr2=0x7f2d640d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d63e00640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d6415f270 0x7f2d640d3750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02a180 comp rx=0 tx=0).stop 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d63e00640 1 -- 172.21.15.106:0/3692243645 shutdown_connections 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d63e00640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d640d1df0 0x7f2d640b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.159+0000 7f2d63e00640 1 --2- 172.21.15.106:0/3692243645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d6415f270 0x7f2d640d3750 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.160+0000 7f2d63e00640 1 -- 172.21.15.106:0/3692243645 >> 172.21.15.106:0/3692243645 conn(0x7f2d6409e2e0 msgr2=0x7f2d640d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.160+0000 7f2d63e00640 1 -- 172.21.15.106:0/3692243645 shutdown_connections 2024-04-26T21:47:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.160+0000 7f2d63e00640 1 -- 172.21.15.106:0/3692243645 wait complete. 2024-04-26T21:47:09.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.161+0000 7f2d63e00640 1 Processor -- start 2024-04-26T21:47:09.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.161+0000 7f2d63e00640 1 -- start start 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d640d3c90 0x7f2d64160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d6415f270 0x7f2d640cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d640d09b0 con 0x7f2d640d1df0 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d640d06d0 con 0x7f2d640d3c90 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.162+0000 7f2d63e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d640cfae0 con 0x7f2d6415f270 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38114/0 (socket says 172.21.15.106:38114) 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 -- 172.21.15.106:0/2009024679 learned_addr learned my addr 172.21.15.106:0/2009024679 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 -- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d6415f270 msgr2=0x7f2d640cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d6415f270 0x7f2d640cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 -- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d640d3c90 msgr2=0x7f2d64160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d640d3c90 0x7f2d64160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 -- 172.21.15.106:0/2009024679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d64160920 con 0x7f2d640d1df0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.163+0000 7f2d62a00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.164+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037040 con 0x7f2d640d1df0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.164+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c042070 con 0x7f2d640d1df0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.164+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c030a70 con 0x7f2d640d1df0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.164+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d64160c00 con 0x7f2d640d1df0 2024-04-26T21:47:09.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.164+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641611c0 con 0x7f2d640d1df0 2024-04-26T21:47:09.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.165+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d640693c0 con 0x7f2d640d1df0 2024-04-26T21:47:09.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.166+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d4c030c10 con 0x7f2d640d1df0 2024-04-26T21:47:09.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.167+0000 7f2d5aa00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.167+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f2d4c0ac240 con 0x7f2d640d1df0 2024-04-26T21:47:09.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.167+0000 7f2d62000640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.168+0000 7f2d62000640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 0x7f2d4006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d54007fd0 tx=0x7f2d54001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.172+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d4c045360 con 0x7f2d640d1df0 2024-04-26T21:47:09.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.444+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f2d640d5670 con 0x7f2d640d1df0 2024-04-26T21:47:09.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.448+0000 7f2d5aa00640 1 -- 172.21.15.106:0/2009024679 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f2d4c07a9b0 con 0x7f2d640d1df0 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.451+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 msgr2=0x7f2d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.451+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 0x7f2d4006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d54007fd0 tx=0x7f2d54001690 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 msgr2=0x7f2d6415b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 shutdown_connections 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d6415f270 0x7f2d640cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d640d3c90 0x7f2d64160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 --2- 172.21.15.106:0/2009024679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d640d1df0 0x7f2d6415b5f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 >> 172.21.15.106:0/2009024679 conn(0x7f2d6409e2e0 msgr2=0x7f2d6406e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 shutdown_connections 2024-04-26T21:47:09.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.452+0000 7f2d63e00640 1 -- 172.21.15.106:0/2009024679 wait complete. 2024-04-26T21:47:09.472 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T21:47:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.667+0000 7f4526800640 1 Processor -- start 2024-04-26T21:47:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.668+0000 7f4526800640 1 -- start start 2024-04-26T21:47:09.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.669+0000 7f4526800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45200cf640 0x7f45200b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.669+0000 7f4526800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.669+0000 7f4526800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45200afb50 con 0x7f45200af670 2024-04-26T21:47:09.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.669+0000 7f4526800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f45200afe30 con 0x7f45200cf640 2024-04-26T21:47:09.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.669+0000 7f4526800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45200b0110 con 0x7f45200b5e60 2024-04-26T21:47:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.670+0000 7f4525e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.670+0000 7f4525e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:60814/0 (socket says 172.21.15.106:60814) 2024-04-26T21:47:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.670+0000 7f4525e00640 1 -- 172.21.15.106:0/1474329118 learned_addr learned my addr 172.21.15.106:0/1474329118 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.670+0000 7f4524a00640 1 --1- 172.21.15.106:0/1474329118 >> v1:172.21.15.204:6789/0 conn(0x7f45200af670 0x7f452015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:42904/0 (socket says 172.21.15.106:42904) 2024-04-26T21:47:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.670+0000 7f4525400640 1 --2- 172.21.15.106:0/1474329118 >> v2:172.21.15.204:3300/0 conn(0x7f45200cf640 0x7f45200b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f451fe00640 1 -- 172.21.15.106:0/1474329118 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743525632 0 0) 0x7f45200afb50 con 0x7f45200af670 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f451fe00640 1 -- 172.21.15.106:0/1474329118 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4508003520 con 0x7f45200af670 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f4525400640 1 -- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 msgr2=0x7f45200d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f4525400640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f4525400640 1 -- 172.21.15.106:0/1474329118 >> v1:172.21.15.204:6789/0 conn(0x7f45200af670 legacy=0x7f452015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f4525400640 1 -- 172.21.15.106:0/1474329118 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45200b03f0 con 0x7f45200cf640 2024-04-26T21:47:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.671+0000 7f4525e00640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.672+0000 7f4525400640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45200b5920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45140097a0 tx=0x7f451402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb1fe19867ab0cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.672+0000 7f451fe00640 1 -- 172.21.15.106:0/1474329118 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514037070 con 0x7f45200cf640 2024-04-26T21:47:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.672+0000 7f451fe00640 1 -- 172.21.15.106:0/1474329118 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4514032440 con 0x7f45200cf640 2024-04-26T21:47:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.672+0000 7f451fe00640 1 -- 172.21.15.106:0/1474329118 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514032810 con 0x7f45200cf640 2024-04-26T21:47:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 -- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 msgr2=0x7f45200b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45200b5920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45140097a0 tx=0x7f451402a180 comp rx=0 tx=0).stop 2024-04-26T21:47:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 -- 172.21.15.106:0/1474329118 shutdown_connections 2024-04-26T21:47:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f45200d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 --2- 172.21.15.106:0/1474329118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45200b5920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 -- 172.21.15.106:0/1474329118 >> 172.21.15.106:0/1474329118 conn(0x7f452006c560 msgr2=0x7f452006c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 -- 172.21.15.106:0/1474329118 shutdown_connections 2024-04-26T21:47:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.673+0000 7f4526800640 1 -- 172.21.15.106:0/1474329118 wait complete. 2024-04-26T21:47:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.675+0000 7f4526800640 1 Processor -- start 2024-04-26T21:47:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.675+0000 7f4526800640 1 -- start start 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f452009e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45201518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45200d1450 con 0x7f45200af670 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45200d12d0 con 0x7f45200cf640 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4526800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45200d15d0 con 0x7f45200b5e60 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4525400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4525400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38128/0 (socket says 172.21.15.106:38128) 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.676+0000 7f4525400640 1 -- 172.21.15.106:0/4163266232 learned_addr learned my addr 172.21.15.106:0/4163266232 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 -- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 msgr2=0x7f452009e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525e00640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45201518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f452009e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 -- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 msgr2=0x7f45201518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45201518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 -- 172.21.15.106:0/4163266232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452009d3f0 con 0x7f45200af670 2024-04-26T21:47:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4525400640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4514035040 tx=0x7f45140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514041030 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451403d4e0 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452009c350 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514037040 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.677+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452009c910 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.679+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452009d030 con 0x7f45200af670 2024-04-26T21:47:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.680+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4514044070 con 0x7f45200af670 2024-04-26T21:47:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.681+0000 7f451d600640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 0x7f450006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.681+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f45140abdf0 con 0x7f45200af670 2024-04-26T21:47:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.681+0000 7f4524a00640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 0x7f450006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.682+0000 7f4524a00640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 0x7f450006ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f450c0011e0 tx=0x7f450c007910 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.685+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f451407a580 con 0x7f45200af670 2024-04-26T21:47:09.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.937+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f452006f680 con 0x7f45200af670 2024-04-26T21:47:09.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.947+0000 7f451d600640 1 -- 172.21.15.106:0/4163266232 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f4514079e80 con 0x7f45200af670 2024-04-26T21:47:09.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 msgr2=0x7f450006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 0x7f450006ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f450c0011e0 tx=0x7f450c007910 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 msgr2=0x7f452009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4514035040 tx=0x7f45140046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 shutdown_connections 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45200cf640 0x7f45201518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f450006c920 0x7f450006ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45200b5e60 0x7f452009e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 --2- 172.21.15.106:0/4163266232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45200af670 0x7f452009ba50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.950+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 >> 172.21.15.106:0/4163266232 conn(0x7f452006c560 msgr2=0x7f45200ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.951+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 shutdown_connections 2024-04-26T21:47:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:09.951+0000 7f4526800640 1 -- 172.21.15.106:0/4163266232 wait complete. 2024-04-26T21:47:09.971 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T21:47:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.168+0000 7f2f88200640 1 Processor -- start 2024-04-26T21:47:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.169+0000 7f2f88200640 1 -- start start 2024-04-26T21:47:10.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.170+0000 7f2f88200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 0x7f2f800add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.170+0000 7f2f88200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800af9a0 con 0x7f2f8015cdf0 2024-04-26T21:47:10.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.170+0000 7f2f88200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800afc80 con 0x7f2f8006ce80 2024-04-26T21:47:10.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.170+0000 7f2f88200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f800ae2c0 con 0x7f2f800cd650 2024-04-26T21:47:10.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.171+0000 7f2f85800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f8015cdf0 0x7f2f8006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38134/0 (socket says 172.21.15.106:38134) 2024-04-26T21:47:10.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.171+0000 7f2f85800640 1 -- 172.21.15.106:0/179607693 learned_addr learned my addr 172.21.15.106:0/179607693 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:10.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.171+0000 7f2f86200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 0x7f2f800add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.171+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160665390 0 0) 0x7f2f800af9a0 con 0x7f2f8015cdf0 2024-04-26T21:47:10.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.171+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f68003520 con 0x7f2f8015cdf0 2024-04-26T21:47:10.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2912792374 0 0) 0x7f2f68003520 con 0x7f2f8015cdf0 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 msgr2=0x7f2f800add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 0x7f2f800add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 >> v1:172.21.15.204:6789/0 conn(0x7f2f8006ce80 legacy=0x7f2f800b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f800aff60 con 0x7f2f8015cdf0 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2f74001670 con 0x7f2f8015cdf0 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 >> v1:172.21.15.106:6789/0 conn(0x7f2f8015cdf0 legacy=0x7f2f8006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f680039e0 0x7f2f68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 0x7f2f68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6800baa0 con 0x7f2f68004d40 2024-04-26T21:47:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6800bc50 con 0x7f2f680039e0 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6800be50 con 0x7f2f800cd650 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f85800640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f680039e0 0x7f2f68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 -- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 msgr2=0x7f2f68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f800cd650 0x7f2f68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 -- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f680039e0 msgr2=0x7f2f68003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f680039e0 0x7f2f68003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f86200640 1 -- 172.21.15.106:0/179607693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f68003520 con 0x7f2f68004d40 2024-04-26T21:47:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.172+0000 7f2f85800640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f680039e0 0x7f2f68003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.173+0000 7f2f86200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009f90 tx=0x7f2f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f069e3e02e3b477d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78009c10 con 0x7f2f68004d40 2024-04-26T21:47:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f78036070 con 0x7f2f68004d40 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f7fe00640 1 -- 172.21.15.106:0/179607693 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78031880 con 0x7f2f68004d40 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f88200640 1 -- 172.21.15.106:0/179607693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 msgr2=0x7f2f680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f88200640 1 --2- 172.21.15.106:0/179607693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f680051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009f90 tx=0x7f2f68005830 comp rx=0 tx=0).stop 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f86200640 1 -- 172.21.15.106:0/179607693 reap_dead start 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f88200640 1 -- 172.21.15.106:0/179607693 shutdown_connections 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.174+0000 7f2f88200640 1 -- 172.21.15.106:0/179607693 >> 172.21.15.106:0/179607693 conn(0x7f2f800b2d50 msgr2=0x7f2f800b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.175+0000 7f2f88200640 1 -- 172.21.15.106:0/179607693 shutdown_connections 2024-04-26T21:47:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.175+0000 7f2f88200640 1 -- 172.21.15.106:0/179607693 wait complete. 2024-04-26T21:47:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.176+0000 7f2f88200640 1 Processor -- start 2024-04-26T21:47:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.176+0000 7f2f88200640 1 -- start start 2024-04-26T21:47:10.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f801f91c0 0x7f2f8015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8009c920 0x7f2f8015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8015d1d0 con 0x7f2f68004d40 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f800b05b0 con 0x7f2f8009c920 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.177+0000 7f2f88200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8015d350 con 0x7f2f801f91c0 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38148/0 (socket says 172.21.15.106:38148) 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 -- 172.21.15.106:0/4257099201 learned_addr learned my addr 172.21.15.106:0/4257099201 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 -- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f801f91c0 msgr2=0x7f2f8015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f86200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8009c920 0x7f2f8015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f801f91c0 0x7f2f8015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 -- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8009c920 msgr2=0x7f2f8015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8009c920 0x7f2f8015b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 -- 172.21.15.106:0/4257099201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8009d270 con 0x7f2f68004d40 2024-04-26T21:47:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.178+0000 7f2f85800640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400d860 tx=0x7f2f7400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.179+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74019070 con 0x7f2f68004d40 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.179+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f740074e0 con 0x7f2f68004d40 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.179+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f740049e0 con 0x7f2f68004d40 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.179+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f801f5090 con 0x7f2f68004d40 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.179+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f801f5650 con 0x7f2f68004d40 2024-04-26T21:47:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.180+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f80069540 con 0x7f2f68004d40 2024-04-26T21:47:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.181+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7401c070 con 0x7f2f68004d40 2024-04-26T21:47:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.182+0000 7f2f7d600640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.182+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f2f7408b420 con 0x7f2f68004d40 2024-04-26T21:47:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.183+0000 7f2f84e00640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.183+0000 7f2f84e00640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 0x7f2f6006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f8009c8d0 tx=0x7f2f6c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.187+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f74059b00 con 0x7f2f68004d40 2024-04-26T21:47:10.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.453+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f2f800b5340 con 0x7f2f68004d40 2024-04-26T21:47:10.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.457+0000 7f2f7d600640 1 -- 172.21.15.106:0/4257099201 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) ==== 97+0+0 (secure 0 0 0) 0x7f2f74059400 con 0x7f2f68004d40 2024-04-26T21:47:10.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 msgr2=0x7f2f6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 0x7f2f6006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f8009c8d0 tx=0x7f2f6c0079b0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 msgr2=0x7f2f801f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400d860 tx=0x7f2f7400dd20 comp rx=0 tx=0).stop 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 shutdown_connections 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8009c920 0x7f2f8015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f801f91c0 0x7f2f8015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 --2- 172.21.15.106:0/4257099201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f68004d40 0x7f2f801f5b70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 >> 172.21.15.106:0/4257099201 conn(0x7f2f800b2d50 msgr2=0x7f2f800b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 shutdown_connections 2024-04-26T21:47:10.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.459+0000 7f2f88200640 1 -- 172.21.15.106:0/4257099201 wait complete. 2024-04-26T21:47:10.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T21:47:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.662+0000 7fbebfe00640 1 Processor -- start 2024-04-26T21:47:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.662+0000 7fbebfe00640 1 -- start start 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 0x7fbec006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebfe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00afe30 con 0x7fbec0072260 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebfe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00b0110 con 0x7fbec00af2e0 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00afc00 con 0x7fbec00d14a0 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebe000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbec0072260 0x7fbec00d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38144/0 (socket says 172.21.15.106:38144) 2024-04-26T21:47:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.664+0000 7fbebe000640 1 -- 172.21.15.106:0/2474398115 learned_addr learned my addr 172.21.15.106:0/2474398115 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916183514 0 0) 0x7fbec00afe30 con 0x7fbec0072260 2024-04-26T21:47:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbea4003520 con 0x7fbec0072260 2024-04-26T21:47:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebf400640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 0x7fbec006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2462519885 0 0) 0x7fbea4003520 con 0x7fbec0072260 2024-04-26T21:47:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 msgr2=0x7fbec006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 0x7fbec006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 >> v1:172.21.15.204:6789/0 conn(0x7fbec00af2e0 legacy=0x7fbec0073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00b03f0 con 0x7fbec0072260 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fbeb0001670 con 0x7fbec0072260 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 >> v1:172.21.15.106:6789/0 conn(0x7fbec0072260 legacy=0x7fbec00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 0x7fbea40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004010 0x7fbea4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbea4004d40 0x7fbea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbea400bb50 con 0x7fbea4004d40 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbea400bd00 con 0x7fbea4004010 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbea400bf00 con 0x7fbec00d14a0 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.665+0000 7fbebf400640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbea4004d40 0x7fbea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebf400640 1 -- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 msgr2=0x7fbea40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebe000640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004010 0x7fbea4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebf400640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec00d14a0 0x7fbea40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebf400640 1 -- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004010 msgr2=0x7fbea4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebf400640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004010 0x7fbea4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebf400640 1 -- 172.21.15.106:0/2474398115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea4003520 con 0x7fbea4004d40 2024-04-26T21:47:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.666+0000 7fbebe000640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004010 0x7fbea4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.667+0000 7fbebf400640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbea4004d40 0x7fbea40051c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007330 tx=0x7fbea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05589c8be3f361d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.667+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8009be0 con 0x7fbea4004d40 2024-04-26T21:47:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.667+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb8036600 con 0x7fbea4004d40 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.667+0000 7fbebd600640 1 -- 172.21.15.106:0/2474398115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80369d0 con 0x7fbea4004d40 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 -- 172.21.15.106:0/2474398115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbea4004d40 msgr2=0x7fbea40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 --2- 172.21.15.106:0/2474398115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbea4004d40 0x7fbea40051c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007330 tx=0x7fbea4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebf400640 1 -- 172.21.15.106:0/2474398115 reap_dead start 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 -- 172.21.15.106:0/2474398115 shutdown_connections 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 -- 172.21.15.106:0/2474398115 >> 172.21.15.106:0/2474398115 conn(0x7fbec006c560 msgr2=0x7fbec006c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 -- 172.21.15.106:0/2474398115 shutdown_connections 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.668+0000 7fbebfe00640 1 -- 172.21.15.106:0/2474398115 wait complete. 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.669+0000 7fbebfe00640 1 Processor -- start 2024-04-26T21:47:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.670+0000 7fbebfe00640 1 -- start start 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 0x7fbec0149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec015bc90 0x7fbec015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00b5dd0 con 0x7fbec0156460 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00b5c50 con 0x7fbea4004d40 2024-04-26T21:47:10.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00af6c0 con 0x7fbec015bc90 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38156/0 (socket says 172.21.15.106:38156) 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 -- 172.21.15.106:0/1836871721 learned_addr learned my addr 172.21.15.106:0/1836871721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 -- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec015bc90 msgr2=0x7fbec015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec015bc90 0x7fbec015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebea00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 0x7fbec0149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 -- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 msgr2=0x7fbec0149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 0x7fbec0149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebe000640 1 -- 172.21.15.106:0/1836871721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00b0880 con 0x7fbec0156460 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.671+0000 7fbebea00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 0x7fbec0149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbebe000640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000da90 tx=0x7fbeb000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0019070 con 0x7fbec0156460 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb00074e0 con 0x7fbec0156460 2024-04-26T21:47:10.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00047e0 con 0x7fbec0156460 2024-04-26T21:47:10.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec00b0b60 con 0x7fbec0156460 2024-04-26T21:47:10.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.672+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec00b1120 con 0x7fbec0156460 2024-04-26T21:47:10.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.673+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec00693c0 con 0x7fbec0156460 2024-04-26T21:47:10.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.674+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbeb0004980 con 0x7fbec0156460 2024-04-26T21:47:10.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.675+0000 7fbeb6a00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.675+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbeb008b3a0 con 0x7fbec0156460 2024-04-26T21:47:10.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.675+0000 7fbebea00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.676+0000 7fbebea00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 0x7fbea006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0016f0 tx=0x7fbeac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.679+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbeb0059b10 con 0x7fbec0156460 2024-04-26T21:47:10.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.955+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fbec00d0240 con 0x7fbec0156460 2024-04-26T21:47:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.960+0000 7fbeb6a00640 1 -- 172.21.15.106:0/1836871721 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fbeb0059410 con 0x7fbec0156460 2024-04-26T21:47:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.962+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 msgr2=0x7fbea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.962+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 0x7fbea006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0016f0 tx=0x7fbeac009040 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.962+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 msgr2=0x7fbec01568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.962+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000da90 tx=0x7fbeb000df50 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 shutdown_connections 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec015bc90 0x7fbec015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec0156460 0x7fbec01568e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 --2- 172.21.15.106:0/1836871721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea4004d40 0x7fbec0149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 >> 172.21.15.106:0/1836871721 conn(0x7fbec006c560 msgr2=0x7fbec009e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 shutdown_connections 2024-04-26T21:47:10.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:10.963+0000 7fbebfe00640 1 -- 172.21.15.106:0/1836871721 wait complete. 2024-04-26T21:47:10.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:11.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.179+0000 7f2543200640 1 Processor -- start 2024-04-26T21:47:11.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.179+0000 7f2543200640 1 -- start start 2024-04-26T21:47:11.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.180+0000 7f2543200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 0x7f2544169e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.180+0000 7f2543200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2544163920 con 0x7f25440d5990 2024-04-26T21:47:11.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.180+0000 7f2543200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25440d3f50 con 0x7f254415f1b0 2024-04-26T21:47:11.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.180+0000 7f2543200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25440d36b0 con 0x7f25440b92c0 2024-04-26T21:47:11.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2541400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25440d5990 0x7f2544163210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38148/0 (socket says 172.21.15.106:38148) 2024-04-26T21:47:11.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2541400640 1 -- 172.21.15.106:0/3470166937 learned_addr learned my addr 172.21.15.106:0/3470166937 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:11.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2542800640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 0x7f2544169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839145085 0 0) 0x7f2544163920 con 0x7f25440d5990 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2524003520 con 0x7f25440d5990 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169573819 0 0) 0x7f2524003520 con 0x7f25440d5990 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 msgr2=0x7f2544169e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 0x7f2544169e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.181+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 >> v1:172.21.15.204:6789/0 conn(0x7f254415f1b0 legacy=0x7f2544162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440d3990 con 0x7f25440d5990 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f252c0019f0 con 0x7f25440d5990 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 >> v1:172.21.15.106:6789/0 conn(0x7f25440d5990 legacy=0x7f2544163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039e0 0x7f2524003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524004800 0x7f2524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 0x7f25240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25440d36b0 con 0x7f2524004800 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f252400baa0 con 0x7f25240039e0 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f252400bca0 con 0x7f25440b92c0 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524004800 0x7f2524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 -- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 msgr2=0x7f25240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440b92c0 0x7f25240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 -- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039e0 msgr2=0x7f2524003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039e0 0x7f2524003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.182+0000 7f2541400640 1 -- 172.21.15.106:0/3470166937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2524003520 con 0x7f2524004800 2024-04-26T21:47:11.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.183+0000 7f2541400640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524004800 0x7f2524004c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f252c002230 tx=0x7f2524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6413f0dea2072ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c00a4a0 con 0x7f2524004800 2024-04-26T21:47:11.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c02f070 con 0x7f2524004800 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2540a00640 1 -- 172.21.15.106:0/3470166937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c02bc80 con 0x7f2524004800 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2543200640 1 -- 172.21.15.106:0/3470166937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524004800 msgr2=0x7f2524004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2543200640 1 --2- 172.21.15.106:0/3470166937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2524004800 0x7f2524004c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f252c002230 tx=0x7f2524005830 comp rx=0 tx=0).stop 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2542800640 1 -- 172.21.15.106:0/3470166937 reap_dead start 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2543200640 1 -- 172.21.15.106:0/3470166937 shutdown_connections 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.184+0000 7f2543200640 1 -- 172.21.15.106:0/3470166937 >> 172.21.15.106:0/3470166937 conn(0x7f254409e230 msgr2=0x7f25440d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.185+0000 7f2543200640 1 -- 172.21.15.106:0/3470166937 shutdown_connections 2024-04-26T21:47:11.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.185+0000 7f2543200640 1 -- 172.21.15.106:0/3470166937 wait complete. 2024-04-26T21:47:11.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.186+0000 7f2543200640 1 Processor -- start 2024-04-26T21:47:11.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.186+0000 7f2543200640 1 -- start start 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2524004800 0x7f25440ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f254414f280 0x7f25440cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25440d33a0 con 0x7f25440cc8e0 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25440b7d90 con 0x7f2524004800 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.187+0000 7f2543200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2544167e80 con 0x7f254414f280 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38178/0 (socket says 172.21.15.106:38178) 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 -- 172.21.15.106:0/457842052 learned_addr learned my addr 172.21.15.106:0/457842052 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 -- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f254414f280 msgr2=0x7f25440cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f254414f280 0x7f25440cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 -- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2524004800 msgr2=0x7f25440ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2524004800 0x7f25440ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 -- 172.21.15.106:0/457842052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254415a360 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.188+0000 7f2542800640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f25440d2970 tx=0x7f253c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.189+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c00c620 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.189+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c01a070 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.189+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c00ca00 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.189+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254415a640 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.189+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254415ac00 con 0x7f25440cc8e0 2024-04-26T21:47:11.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.190+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25440693c0 con 0x7f25440cc8e0 2024-04-26T21:47:11.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.191+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f253c00cba0 con 0x7f25440cc8e0 2024-04-26T21:47:11.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.192+0000 7f253a000640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.192+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f253c08b680 con 0x7f25440cc8e0 2024-04-26T21:47:11.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.192+0000 7f2541e00640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.193+0000 7f2541e00640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 0x7f252006ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2534001590 tx=0x7f2534007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.196+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f253c01c310 con 0x7f25440cc8e0 2024-04-26T21:47:11.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.401+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f254408d090 con 0x7f25440cc8e0 2024-04-26T21:47:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.403+0000 7f253a000640 1 -- 172.21.15.106:0/457842052 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12903 (secure 0 0 0) 0x7f253c059d60 con 0x7f25440cc8e0 2024-04-26T21:47:11.404 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:11.405 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":29,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:07.426304+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 msgr2=0x7f252006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 0x7f252006ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2534001590 tx=0x7f2534007990 comp rx=0 tx=0).stop 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 msgr2=0x7f254415d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f25440d2970 tx=0x7f253c00ec00 comp rx=0 tx=0).stop 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 shutdown_connections 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25440cc8e0 0x7f254415d4e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f254414f280 0x7f25440cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 --2- 172.21.15.106:0/457842052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2524004800 0x7f25440ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 >> 172.21.15.106:0/457842052 conn(0x7f254409e230 msgr2=0x7f254406cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 shutdown_connections 2024-04-26T21:47:11.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.406+0000 7f2543200640 1 -- 172.21.15.106:0/457842052 wait complete. 2024-04-26T21:47:11.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:11.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:11.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.625+0000 7f3dac000640 1 Processor -- start 2024-04-26T21:47:11.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.625+0000 7f3dac000640 1 -- start start 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3dac000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3da40d19c0 0x7f3da40d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3dac000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3da40b6170 0x7f3da40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3dac000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da40d3830 con 0x7f3da40d1da0 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3dac000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3da40d40d0 con 0x7f3da40d19c0 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3dac000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3da40d3b10 con 0x7f3da40b6170 2024-04-26T21:47:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3da9600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3da40d19c0 0x7f3da40d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3da9600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3da40d19c0 0x7f3da40d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38192/0 (socket says 172.21.15.106:38192) 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.627+0000 7f3da9600640 1 -- 172.21.15.106:0/738075632 learned_addr learned my addr 172.21.15.106:0/738075632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.628+0000 7f3da9600640 1 -- 172.21.15.106:0/738075632 >> v2:172.21.15.204:3300/0 conn(0x7f3da40b6170 msgr2=0x7f3da40c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.628+0000 7f3da9600640 1 --2- 172.21.15.106:0/738075632 >> v2:172.21.15.204:3300/0 conn(0x7f3da40b6170 0x7f3da40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.628+0000 7f3da9600640 1 -- 172.21.15.106:0/738075632 >> v1:172.21.15.204:6789/0 conn(0x7f3da40d1da0 legacy=0x7f3da4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.628+0000 7f3da9600640 1 -- 172.21.15.106:0/738075632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da40d3df0 con 0x7f3da40d19c0 2024-04-26T21:47:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.629+0000 7f3da9600640 1 --2- 172.21.15.106:0/738075632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40d19c0 0x7f3da40d2f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3da408ede0 tx=0x7f3d9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91444db1f182ec01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.629+0000 7f3da3e00640 1 -- 172.21.15.106:0/738075632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98037070 con 0x7f3da40d19c0 2024-04-26T21:47:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.629+0000 7f3da3e00640 1 -- 172.21.15.106:0/738075632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98032440 con 0x7f3da40d19c0 2024-04-26T21:47:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.629+0000 7f3da3e00640 1 -- 172.21.15.106:0/738075632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98032810 con 0x7f3da40d19c0 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 -- 172.21.15.106:0/738075632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40d19c0 msgr2=0x7f3da40d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 --2- 172.21.15.106:0/738075632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40d19c0 0x7f3da40d2f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3da408ede0 tx=0x7f3d9802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 -- 172.21.15.106:0/738075632 shutdown_connections 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 --2- 172.21.15.106:0/738075632 >> v2:172.21.15.204:3300/0 conn(0x7f3da40b6170 0x7f3da40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 --2- 172.21.15.106:0/738075632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40d19c0 0x7f3da40d2f00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 -- 172.21.15.106:0/738075632 >> 172.21.15.106:0/738075632 conn(0x7f3da409e460 msgr2=0x7f3da40d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 -- 172.21.15.106:0/738075632 shutdown_connections 2024-04-26T21:47:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.630+0000 7f3dac000640 1 -- 172.21.15.106:0/738075632 wait complete. 2024-04-26T21:47:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.632+0000 7f3dac000640 1 Processor -- start 2024-04-26T21:47:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.632+0000 7f3dac000640 1 -- start start 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 0x7f3da409f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3da40d1da0 0x7f3da414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3da40cfc30 con 0x7f3da40b6170 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3da40c91b0 con 0x7f3da40d19c0 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3dac000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3da4162d50 con 0x7f3da40d1da0 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38200/0 (socket says 172.21.15.106:38200) 2024-04-26T21:47:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 -- 172.21.15.106:0/2503127468 learned_addr learned my addr 172.21.15.106:0/2503127468 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 -- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3da40d1da0 msgr2=0x7f3da414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3da40d1da0 0x7f3da414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da9600640 1 -- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 msgr2=0x7f3da409f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.633+0000 7f3da8c00640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 0x7f3da409f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da9600640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 0x7f3da409f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da9600640 1 -- 172.21.15.106:0/2503127468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da414d9b0 con 0x7f3da40b6170 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da8c00640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 0x7f3da409f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da9600640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3d98000c00 tx=0x7f3d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98041070 con 0x7f3da40b6170 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9803c870 con 0x7f3da40b6170 2024-04-26T21:47:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da414dc90 con 0x7f3da40b6170 2024-04-26T21:47:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98037040 con 0x7f3da40b6170 2024-04-26T21:47:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.634+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da414e250 con 0x7f3da40b6170 2024-04-26T21:47:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.636+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da4069650 con 0x7f3da40b6170 2024-04-26T21:47:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.636+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d98044070 con 0x7f3da40b6170 2024-04-26T21:47:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.637+0000 7f3da1600640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 0x7f3d8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.637+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3d980ac4a0 con 0x7f3da40b6170 2024-04-26T21:47:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.638+0000 7f3da8c00640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 0x7f3d8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.638+0000 7f3da8c00640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 0x7f3d8406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.642+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d9807ad40 con 0x7f3da40b6170 2024-04-26T21:47:11.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.841+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3da4162890 con 0x7f3da40b6170 2024-04-26T21:47:11.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.842+0000 7f3da1600640 1 -- 172.21.15.106:0/2503127468 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f3d9807a640 con 0x7f3da40b6170 2024-04-26T21:47:11.843 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 msgr2=0x7f3d8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 0x7f3d8406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90008040 comp rx=0 tx=0).stop 2024-04-26T21:47:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 msgr2=0x7f3da414cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3d98000c00 tx=0x7f3d980046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 shutdown_connections 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3d8406c920 0x7f3d8406ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3da40d1da0 0x7f3da414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da40d19c0 0x7f3da409f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 --2- 172.21.15.106:0/2503127468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da40b6170 0x7f3da414cc50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 >> 172.21.15.106:0/2503127468 conn(0x7f3da409e460 msgr2=0x7f3da40c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 shutdown_connections 2024-04-26T21:47:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:11.845+0000 7f3dac000640 1 -- 172.21.15.106:0/2503127468 wait complete. 2024-04-26T21:47:11.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:47:12.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.056+0000 7f0b86200640 1 Processor -- start 2024-04-26T21:47:12.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.056+0000 7f0b86200640 1 -- start start 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b86200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0b80073720 0x7f0b8006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b86200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b800701a0 con 0x7f0b8015dec0 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b86200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b8015e320 con 0x7f0b8006ed40 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b86200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8006df80 con 0x7f0b80073720 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b77e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b8015dec0 0x7f0b80073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38162/0 (socket says 172.21.15.106:38162) 2024-04-26T21:47:12.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.058+0000 7f0b77e00640 1 -- 172.21.15.106:0/2312038413 learned_addr learned my addr 172.21.15.106:0/2312038413 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:12.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1780873474 0 0) 0x7f0b800701a0 con 0x7f0b8015dec0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b60003520 con 0x7f0b8015dec0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b84e00640 1 --2- 172.21.15.106:0/2312038413 >> v2:172.21.15.204:3300/0 conn(0x7f0b80073720 0x7f0b8006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2094167954 0 0) 0x7f0b60003520 con 0x7f0b8015dec0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 >> v2:172.21.15.204:3300/0 conn(0x7f0b80073720 msgr2=0x7f0b8006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 --2- 172.21.15.106:0/2312038413 >> v2:172.21.15.204:3300/0 conn(0x7f0b80073720 0x7f0b8006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 >> v1:172.21.15.204:6789/0 conn(0x7f0b8006ed40 legacy=0x7f0b800cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8006e260 con 0x7f0b8015dec0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f0b70001670 con 0x7f0b8015dec0 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 >> v1:172.21.15.106:6789/0 conn(0x7f0b8015dec0 legacy=0x7f0b80073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b600039b0 0x7f0b60003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b60004800 0x7f0b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8006df80 con 0x7f0b600052e0 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6000c1e0 con 0x7f0b600039b0 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.059+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6000c3e0 con 0x7f0b60004800 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b84e00640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b600039b0 0x7f0b60003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 -- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b60004800 msgr2=0x7f0b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b60004800 0x7f0b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 -- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b600039b0 msgr2=0x7f0b60003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b600039b0 0x7f0b60003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b85800640 1 -- 172.21.15.106:0/2312038413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b60003520 con 0x7f0b600052e0 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.060+0000 7f0b84e00640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b600039b0 0x7f0b60003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.061+0000 7f0b85800640 1 -- 172.21.15.106:0/2312038413 reap_dead start 2024-04-26T21:47:12.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.061+0000 7f0b85800640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b6000b9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004480 tx=0x7f0b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78dbcdbd6a015f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.061+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78004080 con 0x7f0b600052e0 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.061+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7802ad80 con 0x7f0b600052e0 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.061+0000 7f0b7fe00640 1 -- 172.21.15.106:0/2312038413 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78006d90 con 0x7f0b600052e0 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 -- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 msgr2=0x7f0b6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b6000b9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004480 tx=0x7f0b60005830 comp rx=0 tx=0).stop 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 -- 172.21.15.106:0/2312038413 shutdown_connections 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 --2- 172.21.15.106:0/2312038413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b6000b9c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 -- 172.21.15.106:0/2312038413 >> 172.21.15.106:0/2312038413 conn(0x7f0b800d0ae0 msgr2=0x7f0b800d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 -- 172.21.15.106:0/2312038413 shutdown_connections 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.062+0000 7f0b86200640 1 -- 172.21.15.106:0/2312038413 wait complete. 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.064+0000 7f0b86200640 1 Processor -- start 2024-04-26T21:47:12.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.064+0000 7f0b86200640 1 -- start start 2024-04-26T21:47:12.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b801446e0 0x7f0b800af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b80144b60 0x7f0b800afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80146910 con 0x7f0b600052e0 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8006f1f0 con 0x7f0b801446e0 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b86200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b80146a90 con 0x7f0b80144b60 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38222/0 (socket says 172.21.15.106:38222) 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.065+0000 7f0b84e00640 1 -- 172.21.15.106:0/3683669619 learned_addr learned my addr 172.21.15.106:0/3683669619 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b77e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b801446e0 0x7f0b800af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b85800640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b80144b60 0x7f0b800afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 -- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b80144b60 msgr2=0x7f0b800afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b80144b60 0x7f0b800afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 -- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b801446e0 msgr2=0x7f0b800af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b801446e0 0x7f0b800af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 -- 172.21.15.106:0/3683669619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800c66e0 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b84e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800d890 tx=0x7f0b6800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68016020 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68004440 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b800c69c0 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.066+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68008890 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.067+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b800c6f80 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.068+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8006e820 con 0x7f0b600052e0 2024-04-26T21:47:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.069+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b68019020 con 0x7f0b600052e0 2024-04-26T21:47:12.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.069+0000 7f0b7d600640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 0x7f0b5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.070+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0b6808bbc0 con 0x7f0b600052e0 2024-04-26T21:47:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.070+0000 7f0b77e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 0x7f0b5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.070+0000 7f0b77e00640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 0x7f0b5c06ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000ff0 tx=0x7f0b70001c70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.074+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b8006e820 con 0x7f0b600052e0 2024-04-26T21:47:12.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.275+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0b800693c0 con 0x7f0b600052e0 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.276+0000 7f0b7d600640 1 -- 172.21.15.106:0/3683669619 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f0b68020020 con 0x7f0b600052e0 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:12.277 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-26T21:47:12.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.278+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 msgr2=0x7f0b5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.278+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 0x7f0b5c06ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000ff0 tx=0x7f0b70001c70 comp rx=0 tx=0).stop 2024-04-26T21:47:12.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 msgr2=0x7f0b801421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800d890 tx=0x7f0b6800dd50 comp rx=0 tx=0).stop 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 shutdown_connections 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b5c06c920 0x7f0b5c06ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b80144b60 0x7f0b800afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b801446e0 0x7f0b800af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 --2- 172.21.15.106:0/3683669619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b600052e0 0x7f0b801421d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 >> 172.21.15.106:0/3683669619 conn(0x7f0b800d0ae0 msgr2=0x7f0b8015f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 shutdown_connections 2024-04-26T21:47:12.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.279+0000 7f0b86200640 1 -- 172.21.15.106:0/3683669619 wait complete. 2024-04-26T21:47:12.298 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:47:12.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:47:12.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.503+0000 7f416f200640 1 Processor -- start 2024-04-26T21:47:12.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.504+0000 7f416f200640 1 -- start start 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4170143470 0x7f417006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4170143850 0x7f41700cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 0x7f41700cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f417006df60 con 0x7f4170143470 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4170146ba0 con 0x7f41700cb6e0 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.505+0000 7f416f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f417006e200 con 0x7f4170143850 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4170143470 0x7f417006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4170143470 0x7f417006fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38234/0 (socket says 172.21.15.106:38234) 2024-04-26T21:47:12.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 -- 172.21.15.106:0/1037384793 learned_addr learned my addr 172.21.15.106:0/1037384793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:12.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416e800640 1 --2- 172.21.15.106:0/1037384793 >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 0x7f41700cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f4167e00640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4170143850 0x7f41700cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 -- 172.21.15.106:0/1037384793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4170143850 msgr2=0x7f41700cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4170143850 0x7f41700cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 -- 172.21.15.106:0/1037384793 >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 msgr2=0x7f41700cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 --2- 172.21.15.106:0/1037384793 >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 0x7f41700cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416de00640 1 -- 172.21.15.106:0/1037384793 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417006e4e0 con 0x7f4170143470 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.506+0000 7f416e800640 1 --2- 172.21.15.106:0/1037384793 >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 0x7f41700cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.507+0000 7f416de00640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f417006fb10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f417006f580 tx=0x7f416002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bccafa9470fe842 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.507+0000 7f416d400640 1 -- 172.21.15.106:0/1037384793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160037070 con 0x7f4170143470 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.507+0000 7f416d400640 1 -- 172.21.15.106:0/1037384793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160032440 con 0x7f4170143470 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416d400640 1 -- 172.21.15.106:0/1037384793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160032810 con 0x7f4170143470 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 -- 172.21.15.106:0/1037384793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 msgr2=0x7f417006fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f417006fb10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f417006f580 tx=0x7f416002a140 comp rx=0 tx=0).stop 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 -- 172.21.15.106:0/1037384793 shutdown_connections 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 --2- 172.21.15.106:0/1037384793 >> v2:172.21.15.204:3300/0 conn(0x7f41700cb6e0 0x7f41700cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4170143850 0x7f41700cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 --2- 172.21.15.106:0/1037384793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f417006fb10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 -- 172.21.15.106:0/1037384793 >> 172.21.15.106:0/1037384793 conn(0x7f41700d0b50 msgr2=0x7f41700d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 -- 172.21.15.106:0/1037384793 shutdown_connections 2024-04-26T21:47:12.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.508+0000 7f416f200640 1 -- 172.21.15.106:0/1037384793 wait complete. 2024-04-26T21:47:12.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.510+0000 7f416f200640 1 Processor -- start 2024-04-26T21:47:12.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.510+0000 7f416f200640 1 -- start start 2024-04-26T21:47:12.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 0x7f417013d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4170143850 0x7f41700b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4170160c20 con 0x7f4170143470 2024-04-26T21:47:12.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4170070180 con 0x7f4170143850 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f416f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4170160da0 con 0x7f41700cb6e0 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f4167e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f4167e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38240/0 (socket says 172.21.15.106:38240) 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.511+0000 7f4167e00640 1 -- 172.21.15.106:0/1647630603 learned_addr learned my addr 172.21.15.106:0/1647630603 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f416e800640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4170143850 0x7f41700b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f416de00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 0x7f417013d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 -- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 msgr2=0x7f417013d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 0x7f417013d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 -- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4170143850 msgr2=0x7f41700b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4170143850 0x7f41700b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 -- 172.21.15.106:0/1647630603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417013dbd0 con 0x7f4170143470 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f416de00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 0x7f417013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4167e00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f415800c7c0 tx=0x7f415800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158015070 con 0x7f4170143470 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4158007820 con 0x7f4170143470 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417013deb0 con 0x7f4170143470 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158007bd0 con 0x7f4170143470 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.512+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417013e470 con 0x7f4170143470 2024-04-26T21:47:12.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.514+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417006d1f0 con 0x7f4170143470 2024-04-26T21:47:12.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.515+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4158018070 con 0x7f4170143470 2024-04-26T21:47:12.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.516+0000 7f4166000640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.516+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f415808b2f0 con 0x7f4170143470 2024-04-26T21:47:12.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.516+0000 7f416de00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.517+0000 7f416de00640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 0x7f414c06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4160034000 tx=0x7f416002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.521+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f417006d1f0 con 0x7f4170143470 2024-04-26T21:47:12.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:12.721+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f41700ccb30 con 0x7f4170143470 2024-04-26T21:47:13.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.433+0000 7f4166000640 1 -- 172.21.15.106:0/1647630603 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f41580599d0 con 0x7f4170143470 2024-04-26T21:47:13.434 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-26T21:47:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.436+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 msgr2=0x7f414c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.436+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 0x7f414c06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4160034000 tx=0x7f416002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:13.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.436+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 msgr2=0x7f41700b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f415800c7c0 tx=0x7f415800cc80 comp rx=0 tx=0).stop 2024-04-26T21:47:13.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 shutdown_connections 2024-04-26T21:47:13.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4170143850 0x7f41700b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4170143470 0x7f41700b3760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 --2- 172.21.15.106:0/1647630603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700cb6e0 0x7f417013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 >> 172.21.15.106:0/1647630603 conn(0x7f41700d0b50 msgr2=0x7f41700cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.437+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 shutdown_connections 2024-04-26T21:47:13.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.438+0000 7f416f200640 1 -- 172.21.15.106:0/1647630603 wait complete. 2024-04-26T21:47:13.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:47:13.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.664+0000 7f206a600640 1 Processor -- start 2024-04-26T21:47:13.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.665+0000 7f206a600640 1 -- start start 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2064070020 0x7f20640d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20640b40c0 0x7f20640b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2064071f30 0x7f20640723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f20640728e0 con 0x7f20640b40c0 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f20640b5790 con 0x7f2064070020 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.666+0000 7f206a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f20640d06a0 con 0x7f2064071f30 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20640b40c0 0x7f20640b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20640b40c0 0x7f20640b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38242/0 (socket says 172.21.15.106:38242) 2024-04-26T21:47:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 -- 172.21.15.106:0/4032029388 learned_addr learned my addr 172.21.15.106:0/4032029388 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2069200640 1 --2- 172.21.15.106:0/4032029388 >> v2:172.21.15.204:3300/0 conn(0x7f2064070020 0x7f20640d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2069c00640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2064071f30 0x7f20640723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 -- 172.21.15.106:0/4032029388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2064071f30 msgr2=0x7f20640723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2064071f30 0x7f20640723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 -- 172.21.15.106:0/4032029388 >> v2:172.21.15.204:3300/0 conn(0x7f2064070020 msgr2=0x7f20640d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 --2- 172.21.15.106:0/4032029388 >> v2:172.21.15.204:3300/0 conn(0x7f2064070020 0x7f20640d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.667+0000 7f2063e00640 1 -- 172.21.15.106:0/4032029388 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20640b4b80 con 0x7f20640b40c0 2024-04-26T21:47:13.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.668+0000 7f2063e00640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20640b40c0 0x7f20640b4550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2064071b50 tx=0x7f205002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a21434dd483fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.668+0000 7f2063400640 1 -- 172.21.15.106:0/4032029388 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050037070 con 0x7f20640b40c0 2024-04-26T21:47:13.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.668+0000 7f2063400640 1 -- 172.21.15.106:0/4032029388 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2050032440 con 0x7f20640b40c0 2024-04-26T21:47:13.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f2063400640 1 -- 172.21.15.106:0/4032029388 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050032810 con 0x7f20640b40c0 2024-04-26T21:47:13.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 -- 172.21.15.106:0/4032029388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20640b40c0 msgr2=0x7f20640b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20640b40c0 0x7f20640b4550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2064071b50 tx=0x7f205002a180 comp rx=0 tx=0).stop 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 -- 172.21.15.106:0/4032029388 shutdown_connections 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2064071f30 0x7f20640723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 --2- 172.21.15.106:0/4032029388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20640b40c0 0x7f20640b4550 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 --2- 172.21.15.106:0/4032029388 >> v2:172.21.15.204:3300/0 conn(0x7f2064070020 0x7f20640d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 -- 172.21.15.106:0/4032029388 >> 172.21.15.106:0/4032029388 conn(0x7f20640b2790 msgr2=0x7f20640b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 -- 172.21.15.106:0/4032029388 shutdown_connections 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.669+0000 7f206a600640 1 -- 172.21.15.106:0/4032029388 wait complete. 2024-04-26T21:47:13.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.671+0000 7f206a600640 1 Processor -- start 2024-04-26T21:47:13.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.671+0000 7f206a600640 1 -- start start 2024-04-26T21:47:13.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2064071f30 0x7f20641498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20640b40c0 0x7f2064149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20640b5420 con 0x7f2064070020 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20640b5140 con 0x7f2064071f30 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f206a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f206415b950 con 0x7f20640b40c0 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f2069200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f2069200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38246/0 (socket says 172.21.15.106:38246) 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.672+0000 7f2069200640 1 -- 172.21.15.106:0/3334541050 learned_addr learned my addr 172.21.15.106:0/3334541050 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 -- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20640b40c0 msgr2=0x7f2064149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2063e00640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2064071f30 0x7f20641498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20640b40c0 0x7f2064149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 -- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2064071f30 msgr2=0x7f20641498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2064071f30 0x7f20641498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 -- 172.21.15.106:0/3334541050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2064152440 con 0x7f2064070020 2024-04-26T21:47:13.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2069200640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f205800d800 tx=0x7f205800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058018020 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20580096a0 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2064152720 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058009a70 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.673+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2064154db0 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.674+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2064050760 con 0x7f2064070020 2024-04-26T21:47:13.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.676+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f205801b020 con 0x7f2064070020 2024-04-26T21:47:13.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.677+0000 7f2060c00640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 0x7f204406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.677+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f205808c3c0 con 0x7f2064070020 2024-04-26T21:47:13.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.677+0000 7f2063e00640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 0x7f204406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.678+0000 7f2063e00640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 0x7f204406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2050034000 tx=0x7f205002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.681+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2058022020 con 0x7f2064070020 2024-04-26T21:47:13.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:13.881+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2064068410 con 0x7f2064070020 2024-04-26T21:47:14.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.440+0000 7f2060c00640 1 -- 172.21.15.106:0/3334541050 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f205805a9b0 con 0x7f2064070020 2024-04-26T21:47:14.441 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.443+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 msgr2=0x7f204406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.443+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 0x7f204406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2050034000 tx=0x7f205002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.443+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 msgr2=0x7f2064146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.443+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f205800d800 tx=0x7f205800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 shutdown_connections 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f204406c8f0 0x7f204406edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20640b40c0 0x7f2064149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2064071f30 0x7f20641498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 --2- 172.21.15.106:0/3334541050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2064070020 0x7f2064146d60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 >> 172.21.15.106:0/3334541050 conn(0x7f20640b2790 msgr2=0x7f206415c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:14.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 shutdown_connections 2024-04-26T21:47:14.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.444+0000 7f206a600640 1 -- 172.21.15.106:0/3334541050 wait complete. 2024-04-26T21:47:14.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:47:14.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.656+0000 7f54a3200640 1 Processor -- start 2024-04-26T21:47:14.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.656+0000 7f54a3200640 1 -- start start 2024-04-26T21:47:14.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a3200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54a40cfb90 0x7f54a40d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a3200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f54a40d2c50 0x7f54a40d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a3200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40d0170 con 0x7f54a415f7f0 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a3200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f54a40d0450 con 0x7f54a40cfb90 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a3200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f54a40d0730 con 0x7f54a40d2c50 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a1400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54a40cfb90 0x7f54a40d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a1e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f54a415f7f0 0x7f54a415fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38168/0 (socket says 172.21.15.106:38168) 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.658+0000 7f54a1e00640 1 -- 172.21.15.106:0/2643627992 learned_addr learned my addr 172.21.15.106:0/2643627992 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.659+0000 7f54a2800640 1 --2- 172.21.15.106:0/2643627992 >> v2:172.21.15.204:3300/0 conn(0x7f54a40d2c50 0x7f54a40d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.659+0000 7f54a1400640 1 -- 172.21.15.106:0/2643627992 >> v2:172.21.15.204:3300/0 conn(0x7f54a40d2c50 msgr2=0x7f54a40d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.659+0000 7f54a1400640 1 --2- 172.21.15.106:0/2643627992 >> v2:172.21.15.204:3300/0 conn(0x7f54a40d2c50 0x7f54a40d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.659+0000 7f54a1400640 1 -- 172.21.15.106:0/2643627992 >> v1:172.21.15.106:6789/0 conn(0x7f54a415f7f0 legacy=0x7f54a415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.659+0000 7f54a1400640 1 -- 172.21.15.106:0/2643627992 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a40d0a10 con 0x7f54a40cfb90 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.660+0000 7f54a1400640 1 --2- 172.21.15.106:0/2643627992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40d22d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54a40d5fd0 tx=0x7f548c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866c42e2418750c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.660+0000 7f54a0a00640 1 -- 172.21.15.106:0/2643627992 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c007740 con 0x7f54a40cfb90 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.660+0000 7f54a0a00640 1 -- 172.21.15.106:0/2643627992 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c0078e0 con 0x7f54a40cfb90 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.660+0000 7f54a0a00640 1 -- 172.21.15.106:0/2643627992 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c007cb0 con 0x7f54a40cfb90 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 -- 172.21.15.106:0/2643627992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 msgr2=0x7f54a40d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 --2- 172.21.15.106:0/2643627992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40d22d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54a40d5fd0 tx=0x7f548c02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 -- 172.21.15.106:0/2643627992 shutdown_connections 2024-04-26T21:47:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 --2- 172.21.15.106:0/2643627992 >> v2:172.21.15.204:3300/0 conn(0x7f54a40d2c50 0x7f54a40d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 --2- 172.21.15.106:0/2643627992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40d22d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 -- 172.21.15.106:0/2643627992 >> 172.21.15.106:0/2643627992 conn(0x7f54a40d4f80 msgr2=0x7f54a40d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 -- 172.21.15.106:0/2643627992 shutdown_connections 2024-04-26T21:47:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.661+0000 7f54a3200640 1 -- 172.21.15.106:0/2643627992 wait complete. 2024-04-26T21:47:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.663+0000 7f54a3200640 1 Processor -- start 2024-04-26T21:47:14.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.663+0000 7f54a3200640 1 -- start start 2024-04-26T21:47:14.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a40d2c50 0x7f54a4148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54a415f7f0 0x7f54a4148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415f430 con 0x7f54a40cfb90 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54a40d2810 con 0x7f54a40d2c50 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f54a415f5b0 con 0x7f54a415f7f0 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38250/0 (socket says 172.21.15.106:38250) 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a1e00640 1 -- 172.21.15.106:0/793992294 learned_addr learned my addr 172.21.15.106:0/793992294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a1e00640 1 -- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54a415f7f0 msgr2=0x7f54a4148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.664+0000 7f54a1400640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a40d2c50 0x7f54a4148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a1e00640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54a415f7f0 0x7f54a4148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a1e00640 1 -- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a40d2c50 msgr2=0x7f54a4148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a1e00640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a40d2c50 0x7f54a4148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a1e00640 1 -- 172.21.15.106:0/793992294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a40c9c00 con 0x7f54a40cfb90 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a1e00640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f549400c400 tx=0x7f549400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494017070 con 0x7f54a40cfb90 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5494005ce0 con 0x7f54a40cfb90 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a40c9ee0 con 0x7f54a40cfb90 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940072f0 con 0x7f54a40cfb90 2024-04-26T21:47:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.665+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a40ca4a0 con 0x7f54a40cfb90 2024-04-26T21:47:14.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.666+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a40693c0 con 0x7f54a40cfb90 2024-04-26T21:47:14.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.668+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f549401a070 con 0x7f54a40cfb90 2024-04-26T21:47:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.669+0000 7f549a000640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 0x7f548006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.669+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f549408b2c0 con 0x7f54a40cfb90 2024-04-26T21:47:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.669+0000 7f54a1400640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 0x7f548006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.670+0000 7f54a1400640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 0x7f548006edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f548c034000 tx=0x7f548c033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:14.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.673+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5494021020 con 0x7f54a40cfb90 2024-04-26T21:47:14.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:14.906+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f54a415a040 con 0x7f54a40cfb90 2024-04-26T21:47:15.357 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:15.365 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:15.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.481+0000 7f549a000640 1 -- 172.21.15.106:0/793992294 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f5494059830 con 0x7f54a40cfb90 2024-04-26T21:47:15.486 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T21:47:15.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.488+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 msgr2=0x7f548006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.488+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 0x7f548006edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f548c034000 tx=0x7f548c033040 comp rx=0 tx=0).stop 2024-04-26T21:47:15.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.488+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 msgr2=0x7f54a40caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.488+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f549400c400 tx=0x7f549400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 shutdown_connections 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f548006c8f0 0x7f548006edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54a415f7f0 0x7f54a4148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a40d2c50 0x7f54a4148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 --2- 172.21.15.106:0/793992294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54a40cfb90 0x7f54a40caca0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.489+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 >> 172.21.15.106:0/793992294 conn(0x7f54a40d4f80 msgr2=0x7f54a4162370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.490+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 shutdown_connections 2024-04-26T21:47:15.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.490+0000 7f54a3200640 1 -- 172.21.15.106:0/793992294 wait complete. 2024-04-26T21:47:15.525 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.740+0000 7f6a49400640 1 Processor -- start 2024-04-26T21:47:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.740+0000 7f6a49400640 1 -- start start 2024-04-26T21:47:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.741+0000 7f6a49400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.741+0000 7f6a49400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6a4406ce80 0x7f6a440b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.741+0000 7f6a49400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440af9a0 con 0x7f6a440cd650 2024-04-26T21:47:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.741+0000 7f6a49400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440ae2c0 con 0x7f6a4406ce80 2024-04-26T21:47:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.741+0000 7f6a49400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a440b05b0 con 0x7f6a4415cdf0 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.742+0000 7f6a42800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.742+0000 7f6a42800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:60952/0 (socket says 172.21.15.106:60952) 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.742+0000 7f6a42800640 1 -- 172.21.15.106:0/2415542803 learned_addr learned my addr 172.21.15.106:0/2415542803 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.742+0000 7f6a43200640 1 --2- 172.21.15.106:0/2415542803 >> v2:172.21.15.204:3300/0 conn(0x7f6a4406ce80 0x7f6a440b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.743+0000 7f6a42800640 1 -- 172.21.15.106:0/2415542803 >> v2:172.21.15.204:3300/0 conn(0x7f6a4406ce80 msgr2=0x7f6a440b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.743+0000 7f6a42800640 1 --2- 172.21.15.106:0/2415542803 >> v2:172.21.15.204:3300/0 conn(0x7f6a4406ce80 0x7f6a440b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.743+0000 7f6a42800640 1 -- 172.21.15.106:0/2415542803 >> v1:172.21.15.204:6789/0 conn(0x7f6a440cd650 legacy=0x7f6a4406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.743+0000 7f6a42800640 1 -- 172.21.15.106:0/2415542803 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440afc80 con 0x7f6a4415cdf0 2024-04-26T21:47:15.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.744+0000 7f6a42800640 1 --2- 172.21.15.106:0/2415542803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009870 tx=0x7f6a3402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53dcb1862cdba37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a41e00640 1 -- 172.21.15.106:0/2415542803 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037070 con 0x7f6a4415cdf0 2024-04-26T21:47:15.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a41e00640 1 -- 172.21.15.106:0/2415542803 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34032440 con 0x7f6a4415cdf0 2024-04-26T21:47:15.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a41e00640 1 -- 172.21.15.106:0/2415542803 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34032810 con 0x7f6a4415cdf0 2024-04-26T21:47:15.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 -- 172.21.15.106:0/2415542803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 msgr2=0x7f6a440add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 --2- 172.21.15.106:0/2415542803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009870 tx=0x7f6a3402a270 comp rx=0 tx=0).stop 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 -- 172.21.15.106:0/2415542803 shutdown_connections 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 --2- 172.21.15.106:0/2415542803 >> v2:172.21.15.204:3300/0 conn(0x7f6a4406ce80 0x7f6a440b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 --2- 172.21.15.106:0/2415542803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a440add80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.745+0000 7f6a49400640 1 -- 172.21.15.106:0/2415542803 >> 172.21.15.106:0/2415542803 conn(0x7f6a440b2d50 msgr2=0x7f6a440b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.746+0000 7f6a49400640 1 -- 172.21.15.106:0/2415542803 shutdown_connections 2024-04-26T21:47:15.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.746+0000 7f6a49400640 1 -- 172.21.15.106:0/2415542803 wait complete. 2024-04-26T21:47:15.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.747+0000 7f6a49400640 1 Processor -- start 2024-04-26T21:47:15.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.747+0000 7f6a49400640 1 -- start start 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406ce80 0x7f6a441f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a44156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4415d1d0 con 0x7f6a440cd650 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440b0240 con 0x7f6a4406ce80 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.748+0000 7f6a49400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4415d350 con 0x7f6a4415cdf0 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38258/0 (socket says 172.21.15.106:38258) 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 -- 172.21.15.106:0/1294135595 learned_addr learned my addr 172.21.15.106:0/1294135595 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a43200640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a44156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a42800640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406ce80 0x7f6a441f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 -- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 msgr2=0x7f6a44156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a44156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 -- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406ce80 msgr2=0x7f6a441f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406ce80 0x7f6a441f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 -- 172.21.15.106:0/1294135595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440d1770 con 0x7f6a440cd650 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.749+0000 7f6a39e00640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c400 tx=0x7f6a2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.750+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0052d0 con 0x7f6a440cd650 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.750+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2c005490 con 0x7f6a440cd650 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.750+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007260 con 0x7f6a440cd650 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.750+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a441f2c50 con 0x7f6a440cd650 2024-04-26T21:47:15.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.750+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441f3210 con 0x7f6a440cd650 2024-04-26T21:47:15.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.751+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44069540 con 0x7f6a440cd650 2024-04-26T21:47:15.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.752+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a2c007400 con 0x7f6a440cd650 2024-04-26T21:47:15.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.753+0000 7f6a3b400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 0x7f6a2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.753+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6a2c08b0f0 con 0x7f6a440cd650 2024-04-26T21:47:15.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.753+0000 7f6a42800640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 0x7f6a2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.754+0000 7f6a42800640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 0x7f6a2006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a34034000 tx=0x7f6a3402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.758+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a440d1770 con 0x7f6a440cd650 2024-04-26T21:47:15.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.947+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a44069db0 con 0x7f6a440cd650 2024-04-26T21:47:15.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.950+0000 7f6a3b400640 1 -- 172.21.15.106:0/1294135595 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7f6a2c059510 con 0x7f6a440cd650 2024-04-26T21:47:15.952 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:15.952 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:14.908673+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 msgr2=0x7f6a2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 0x7f6a2006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a34034000 tx=0x7f6a3402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 msgr2=0x7f6a440d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c400 tx=0x7f6a2c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 shutdown_connections 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6a2006c8f0 0x7f6a2006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a4415cdf0 0x7f6a44156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a440cd650 0x7f6a440d1230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 --2- 172.21.15.106:0/1294135595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406ce80 0x7f6a441f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 >> 172.21.15.106:0/1294135595 conn(0x7f6a440b2d50 msgr2=0x7f6a4415afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 shutdown_connections 2024-04-26T21:47:15.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:15.953+0000 7f6a49400640 1 -- 172.21.15.106:0/1294135595 wait complete. 2024-04-26T21:47:15.972 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:16.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.165+0000 7fbe59c00640 1 Processor -- start 2024-04-26T21:47:16.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.165+0000 7fbe59c00640 1 -- start start 2024-04-26T21:47:16.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe59c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbe541598a0 0x7fbe541638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe59c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe540b5fb0 con 0x7fbe5406c770 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe59c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe54160370 con 0x7fbe5415fe50 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe59c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbe54159c80 con 0x7fbe541598a0 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe52000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbe5406c770 0x7fbe5415f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38184/0 (socket says 172.21.15.106:38184) 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.167+0000 7fbe52000640 1 -- 172.21.15.106:0/107485553 learned_addr learned my addr 172.21.15.106:0/107485553 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1812165151 0 0) 0x7fbe540b5fb0 con 0x7fbe5406c770 2024-04-26T21:47:16.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe38003520 con 0x7fbe5406c770 2024-04-26T21:47:16.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe52a00640 1 --2- 172.21.15.106:0/107485553 >> v2:172.21.15.204:3300/0 conn(0x7fbe541598a0 0x7fbe541638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 942406067 0 0) 0x7fbe38003520 con 0x7fbe5406c770 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 >> v2:172.21.15.204:3300/0 conn(0x7fbe541598a0 msgr2=0x7fbe541638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 --2- 172.21.15.106:0/107485553 >> v2:172.21.15.204:3300/0 conn(0x7fbe541598a0 0x7fbe541638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 >> v1:172.21.15.204:6789/0 conn(0x7fbe5415fe50 legacy=0x7fbe540add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe54162dd0 con 0x7fbe5406c770 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fbe40001670 con 0x7fbe5406c770 2024-04-26T21:47:16.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 >> v1:172.21.15.106:6789/0 conn(0x7fbe5406c770 legacy=0x7fbe5415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe380039b0 0x7fbe38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe380052e0 0x7fbe3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54159c80 con 0x7fbe38004800 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3800c1b0 con 0x7fbe380039b0 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3800c3b0 con 0x7fbe380052e0 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.168+0000 7fbe52000640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52a00640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe380039b0 0x7fbe38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe53400640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe380052e0 0x7fbe3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52000640 1 -- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe380052e0 msgr2=0x7fbe3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52000640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe380052e0 0x7fbe3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52000640 1 -- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe380039b0 msgr2=0x7fbe38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52000640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe380039b0 0x7fbe38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe52000640 1 -- 172.21.15.106:0/107485553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe38003520 con 0x7fbe38004800 2024-04-26T21:47:16.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe53400640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe380052e0 0x7fbe3800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:16.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.169+0000 7fbe53400640 1 -- 172.21.15.106:0/107485553 reap_dead start 2024-04-26T21:47:16.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.170+0000 7fbe52000640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe38004c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c210 tx=0x7fbe38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf062cb6c0e7473 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.170+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40007cf0 con 0x7fbe38004800 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.170+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe40005ad0 con 0x7fbe38004800 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.170+0000 7fbe51600640 1 -- 172.21.15.106:0/107485553 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40007070 con 0x7fbe38004800 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 -- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 msgr2=0x7fbe38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe38004c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c210 tx=0x7fbe38005830 comp rx=0 tx=0).stop 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 -- 172.21.15.106:0/107485553 shutdown_connections 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 --2- 172.21.15.106:0/107485553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe38004c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 -- 172.21.15.106:0/107485553 >> 172.21.15.106:0/107485553 conn(0x7fbe540b2d50 msgr2=0x7fbe540b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 -- 172.21.15.106:0/107485553 shutdown_connections 2024-04-26T21:47:16.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.171+0000 7fbe59c00640 1 -- 172.21.15.106:0/107485553 wait complete. 2024-04-26T21:47:16.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.173+0000 7fbe59c00640 1 Processor -- start 2024-04-26T21:47:16.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.173+0000 7fbe59c00640 1 -- start start 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5406f3b0 0x7fbe5406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe54143f20 0x7fbe541443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54160650 con 0x7fbe38004800 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54163390 con 0x7fbe54143f20 2024-04-26T21:47:16.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe59c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe541638b0 con 0x7fbe5406f3b0 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38282/0 (socket says 172.21.15.106:38282) 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 -- 172.21.15.106:0/322704521 learned_addr learned my addr 172.21.15.106:0/322704521 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 -- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5406f3b0 msgr2=0x7fbe5406f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5406f3b0 0x7fbe5406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 -- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe54143f20 msgr2=0x7fbe541443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe54143f20 0x7fbe541443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.174+0000 7fbe52a00640 1 -- 172.21.15.106:0/322704521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe541448f0 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe52a00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400f630 tx=0x7fbe4400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44019020 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe44014870 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44014c20 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe540ce8b0 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.175+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe540cee70 con 0x7fbe38004800 2024-04-26T21:47:16.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.176+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54069540 con 0x7fbe38004800 2024-04-26T21:47:16.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.177+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe4401c020 con 0x7fbe38004800 2024-04-26T21:47:16.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.178+0000 7fbe4aa00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.179+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbe4408c4c0 con 0x7fbe38004800 2024-04-26T21:47:16.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.179+0000 7fbe52000640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.180+0000 7fbe52000640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 0x7fbe3406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe40032000 tx=0x7fbe40007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.183+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe44023020 con 0x7fbe38004800 2024-04-26T21:47:16.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.380+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe540d04d0 con 0x7fbe38004800 2024-04-26T21:47:16.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.383+0000 7fbe4aa00640 1 -- 172.21.15.106:0/322704521 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16694 (secure 0 0 0) 0x7fbe44026350 con 0x7fbe38004800 2024-04-26T21:47:16.385 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:16.385 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:14.908673+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:16.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 msgr2=0x7fbe3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 0x7fbe3406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe40032000 tx=0x7fbe40007000 comp rx=0 tx=0).stop 2024-04-26T21:47:16.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 msgr2=0x7fbe540afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400f630 tx=0x7fbe4400faf0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 shutdown_connections 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe54143f20 0x7fbe541443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5406f3b0 0x7fbe5406f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 --2- 172.21.15.106:0/322704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe38004800 0x7fbe540afa60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 >> 172.21.15.106:0/322704521 conn(0x7fbe540b2d50 msgr2=0x7fbe540d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 shutdown_connections 2024-04-26T21:47:16.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.386+0000 7fbe59c00640 1 -- 172.21.15.106:0/322704521 wait complete. 2024-04-26T21:47:16.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:47:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.609+0000 7f1254a00640 1 Processor -- start 2024-04-26T21:47:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.609+0000 7f1254a00640 1 -- start start 2024-04-26T21:47:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.610+0000 7f1254a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f12500d3780 0x7f12500d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.610+0000 7f1254a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f12500b3510 0x7f12500d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f1254a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12500d40a0 con 0x7f12500d58b0 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f1254a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f12500b2070 con 0x7f12500d3780 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f1254a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f12500d0960 con 0x7f12500b3510 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f12500d3780 0x7f12500d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124d400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f12500d58b0 0x7f12500d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38186/0 (socket says 172.21.15.106:38186) 2024-04-26T21:47:16.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124d400640 1 -- 172.21.15.106:0/2981271056 learned_addr learned my addr 172.21.15.106:0/2981271056 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124de00640 1 -- 172.21.15.106:0/2981271056 >> v2:172.21.15.204:3300/0 conn(0x7f12500b3510 msgr2=0x7f12500d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124de00640 1 --2- 172.21.15.106:0/2981271056 >> v2:172.21.15.204:3300/0 conn(0x7f12500b3510 0x7f12500d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124de00640 1 -- 172.21.15.106:0/2981271056 >> v1:172.21.15.106:6789/0 conn(0x7f12500d58b0 legacy=0x7f12500d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124de00640 1 -- 172.21.15.106:0/2981271056 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500d0c40 con 0x7f12500d3780 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.611+0000 7f124ca00640 1 -- 172.21.15.106:0/2981271056 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352337380 0 0) 0x7f12500d40a0 con 0x7f12500d58b0 2024-04-26T21:47:16.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.612+0000 7f124de00640 1 --2- 172.21.15.106:0/2981271056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f12500d3b60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12500cf780 tx=0x7f124002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788943379514986d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f124ca00640 1 -- 172.21.15.106:0/2981271056 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240037070 con 0x7f12500d3780 2024-04-26T21:47:16.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f124ca00640 1 -- 172.21.15.106:0/2981271056 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240032440 con 0x7f12500d3780 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f124ca00640 1 -- 172.21.15.106:0/2981271056 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240032810 con 0x7f12500d3780 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 -- 172.21.15.106:0/2981271056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 msgr2=0x7f12500d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 --2- 172.21.15.106:0/2981271056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f12500d3b60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12500cf780 tx=0x7f124002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 -- 172.21.15.106:0/2981271056 shutdown_connections 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 --2- 172.21.15.106:0/2981271056 >> v2:172.21.15.204:3300/0 conn(0x7f12500b3510 0x7f12500d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 --2- 172.21.15.106:0/2981271056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f12500d3b60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 -- 172.21.15.106:0/2981271056 >> 172.21.15.106:0/2981271056 conn(0x7f125009e4e0 msgr2=0x7f12500d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 -- 172.21.15.106:0/2981271056 shutdown_connections 2024-04-26T21:47:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.613+0000 7f1254a00640 1 -- 172.21.15.106:0/2981271056 wait complete. 2024-04-26T21:47:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.615+0000 7f1254a00640 1 Processor -- start 2024-04-26T21:47:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.615+0000 7f1254a00640 1 -- start start 2024-04-26T21:47:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500b3510 0x7f1250150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500d58b0 0x7f12500c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f12500d32c0 con 0x7f12500d3780 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12500d3140 con 0x7f12500d58b0 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f1254a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f125015f870 con 0x7f12500b3510 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.616+0000 7f124d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38306/0 (socket says 172.21.15.106:38306) 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 -- 172.21.15.106:0/3859426827 learned_addr learned my addr 172.21.15.106:0/3859426827 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124de00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500b3510 0x7f1250150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124e800640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500d58b0 0x7f12500c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 -- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500b3510 msgr2=0x7f1250150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500b3510 0x7f1250150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 -- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500d58b0 msgr2=0x7f12500c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500d58b0 0x7f12500c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 -- 172.21.15.106:0/3859426827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500cf040 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.617+0000 7f124d400640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f123800c4d0 tx=0x7f123800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.618+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238018070 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.618+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238007500 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.618+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238004a20 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.618+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12500b7f80 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.618+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12500cf320 con 0x7f12500d3780 2024-04-26T21:47:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.619+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250069650 con 0x7f12500d3780 2024-04-26T21:47:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.620+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1238004bc0 con 0x7f12500d3780 2024-04-26T21:47:16.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.621+0000 7f1246000640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 0x7f122c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.621+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f123808aee0 con 0x7f12500d3780 2024-04-26T21:47:16.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.621+0000 7f124de00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 0x7f122c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.622+0000 7f124de00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 0x7f122c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1240034000 tx=0x7f124002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.625+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1238059270 con 0x7f12500d3780 2024-04-26T21:47:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:16.865+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f12500d2a90 con 0x7f12500d3780 2024-04-26T21:47:17.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.496+0000 7f1246000640 1 -- 172.21.15.106:0/3859426827 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f1238058b70 con 0x7f12500d3780 2024-04-26T21:47:17.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.499+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 msgr2=0x7f122c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.499+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 0x7f122c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1240034000 tx=0x7f124002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:17.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.499+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 msgr2=0x7f1250150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.499+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f123800c4d0 tx=0x7f123800c990 comp rx=0 tx=0).stop 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 shutdown_connections 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500d58b0 0x7f12500c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12500d3780 0x7f1250150690 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f122c06c8f0 0x7f122c06edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 --2- 172.21.15.106:0/3859426827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500b3510 0x7f1250150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 >> 172.21.15.106:0/3859426827 conn(0x7f125009e4e0 msgr2=0x7f12500b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 shutdown_connections 2024-04-26T21:47:17.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.500+0000 7f1254a00640 1 -- 172.21.15.106:0/3859426827 wait complete. 2024-04-26T21:47:17.523 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:47:17.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:47:17.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.729+0000 7f03b7200640 1 Processor -- start 2024-04-26T21:47:17.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.729+0000 7f03b7200640 1 -- start start 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f03b015ee60 0x7f03b00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f03b006dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b7200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b015d970 con 0x7f03b015c450 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f03b006e520 con 0x7f03b015ee60 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f03b00b4fe0 con 0x7f03b015cf40 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b5e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f03b015c450 0x7f03b015c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38198/0 (socket says 172.21.15.106:38198) 2024-04-26T21:47:17.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b5e00640 1 -- 172.21.15.106:0/1156238463 learned_addr learned my addr 172.21.15.106:0/1156238463 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b5400640 1 --2- 172.21.15.106:0/1156238463 >> v2:172.21.15.204:3300/0 conn(0x7f03b015ee60 0x7f03b00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.731+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273340761 0 0) 0x7f03b015d970 con 0x7f03b015c450 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b6800640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f03b006dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0398003520 con 0x7f03b015c450 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2699654413 0 0) 0x7f0398003520 con 0x7f03b015c450 2024-04-26T21:47:17.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 msgr2=0x7f03b006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f03b006dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 >> v2:172.21.15.204:3300/0 conn(0x7f03b015ee60 msgr2=0x7f03b00b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 --2- 172.21.15.106:0/1156238463 >> v2:172.21.15.204:3300/0 conn(0x7f03b015ee60 0x7f03b00b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b00b52c0 con 0x7f03b015c450 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f03a4001670 con 0x7f03b015c450 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 >> v1:172.21.15.106:6789/0 conn(0x7f03b015c450 legacy=0x7f03b015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0398003a20 0x7f0398003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f0398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0398004d40 0x7f03980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03b006e520 con 0x7f0398004d40 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00b4fe0 con 0x7f0398003a20 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f039800bb20 con 0x7f03b015cf40 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.732+0000 7f03b6800640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0398004d40 0x7f03980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b5e00640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0398003a20 0x7f0398003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b5400640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f0398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b6800640 1 -- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 msgr2=0x7f0398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b6800640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b015cf40 0x7f0398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b6800640 1 -- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0398003a20 msgr2=0x7f0398003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b6800640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0398003a20 0x7f0398003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.733+0000 7f03b6800640 1 -- 172.21.15.106:0/1156238463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0398003520 con 0x7f0398004d40 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.734+0000 7f03b6800640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0398004d40 0x7f03980051c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007360 tx=0x7f0398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ea35e3574a6e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.734+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac009be0 con 0x7f0398004d40 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.734+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac036600 con 0x7f0398004d40 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.734+0000 7f03b4a00640 1 -- 172.21.15.106:0/1156238463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0369d0 con 0x7f0398004d40 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 -- 172.21.15.106:0/1156238463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0398004d40 msgr2=0x7f03980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 --2- 172.21.15.106:0/1156238463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0398004d40 0x7f03980051c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007360 tx=0x7f0398005830 comp rx=0 tx=0).stop 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b6800640 1 -- 172.21.15.106:0/1156238463 reap_dead start 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 -- 172.21.15.106:0/1156238463 shutdown_connections 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 -- 172.21.15.106:0/1156238463 >> 172.21.15.106:0/1156238463 conn(0x7f03b00b2900 msgr2=0x7f03b00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 -- 172.21.15.106:0/1156238463 shutdown_connections 2024-04-26T21:47:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.735+0000 7f03b7200640 1 -- 172.21.15.106:0/1156238463 wait complete. 2024-04-26T21:47:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.737+0000 7f03b7200640 1 Processor -- start 2024-04-26T21:47:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.737+0000 7f03b7200640 1 -- start start 2024-04-26T21:47:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b01416b0 0x7f03b0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 0x7f03b014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00b5a00 con 0x7f03b00cf5d0 2024-04-26T21:47:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00b5880 con 0x7f03b014e490 2024-04-26T21:47:17.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f03b006dfe0 con 0x7f03b01416b0 2024-04-26T21:47:17.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.738+0000 7f03b6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38328/0 (socket says 172.21.15.106:38328) 2024-04-26T21:47:17.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 -- 172.21.15.106:0/992125900 learned_addr learned my addr 172.21.15.106:0/992125900 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b5400640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 0x7f03b014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 -- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b01416b0 msgr2=0x7f03b0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b5e00640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b01416b0 0x7f03b0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b01416b0 0x7f03b0154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 -- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 msgr2=0x7f03b014fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 0x7f03b014fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 -- 172.21.15.106:0/992125900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b014e8f0 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b5400640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 0x7f03b014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.739+0000 7f03b6800640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007450 tx=0x7f03ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.740+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac009be0 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.740+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac036400 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.740+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b014ebc0 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.740+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac038520 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.740+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b014f180 con 0x7f03b00cf5d0 2024-04-26T21:47:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.741+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b00693c0 con 0x7f03b00cf5d0 2024-04-26T21:47:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.742+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03ac02e650 con 0x7f03b00cf5d0 2024-04-26T21:47:17.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.743+0000 7f03aa000640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.743+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f03ac0ac580 con 0x7f03b00cf5d0 2024-04-26T21:47:17.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.743+0000 7f03b5e00640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.744+0000 7f03b5e00640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 0x7f039406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001eb0 tx=0x7f03a40047e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.748+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03ac048060 con 0x7f03b00cf5d0 2024-04-26T21:47:17.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:17.985+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f03b015cae0 con 0x7f03b00cf5d0 2024-04-26T21:47:18.423 INFO:tasks.ceph.mds.b.smithi204.stderr:2024-04-26T21:47:18.422+0000 7f3a02000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.508+0000 7f03aa000640 1 -- 172.21.15.106:0/992125900 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f03ac032030 con 0x7f03b00cf5d0 2024-04-26T21:47:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 msgr2=0x7f039406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 0x7f039406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001eb0 tx=0x7f03a40047e0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 msgr2=0x7f03b00cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007450 tx=0x7f03ac004ab0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 shutdown_connections 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03b00cf5d0 0x7f03b00cfa50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b014e490 0x7f03b014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 --2- 172.21.15.106:0/992125900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03b01416b0 0x7f03b0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.511+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 >> 172.21.15.106:0/992125900 conn(0x7f03b00b2900 msgr2=0x7f03b015dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.512+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 shutdown_connections 2024-04-26T21:47:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.512+0000 7f03b7200640 1 -- 172.21.15.106:0/992125900 wait complete. 2024-04-26T21:47:18.535 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:47:18.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:18.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.742+0000 7f45c4600640 1 Processor -- start 2024-04-26T21:47:18.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.742+0000 7f45c4600640 1 -- start start 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c4600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 0x7f45bc06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c4600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45bc15cfd0 0x7f45bc073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c4600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc06d090 con 0x7f45bc15cb70 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c4600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc0af460 con 0x7f45bc15cfd0 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c4600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc0ae850 con 0x7f45bc06c770 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c2600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45bc15cfd0 0x7f45bc073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c2600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45bc15cfd0 0x7f45bc073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53482/0 (socket says 172.21.15.106:53482) 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.744+0000 7f45c2600640 1 -- 172.21.15.106:0/976574644 learned_addr learned my addr 172.21.15.106:0/976574644 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c2600640 1 -- 172.21.15.106:0/976574644 >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 msgr2=0x7f45bc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c1c00640 1 --2- 172.21.15.106:0/976574644 >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 0x7f45bc06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c2600640 1 --2- 172.21.15.106:0/976574644 >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 0x7f45bc06cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c2600640 1 -- 172.21.15.106:0/976574644 >> v1:172.21.15.204:6789/0 conn(0x7f45bc15cb70 legacy=0x7f45bc073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c2600640 1 -- 172.21.15.106:0/976574644 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0aeb30 con 0x7f45bc15cfd0 2024-04-26T21:47:18.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.745+0000 7f45c1c00640 1 --2- 172.21.15.106:0/976574644 >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 0x7f45bc06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:18.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.746+0000 7f45c2600640 1 --2- 172.21.15.106:0/976574644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cfd0 0x7f45bc073830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0741f0 tx=0x7f45b402c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e6f190aa62b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.746+0000 7f45bbe00640 1 -- 172.21.15.106:0/976574644 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b2d0 con 0x7f45bc15cfd0 2024-04-26T21:47:18.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.746+0000 7f45bbe00640 1 -- 172.21.15.106:0/976574644 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b400b470 con 0x7f45bc15cfd0 2024-04-26T21:47:18.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.746+0000 7f45bbe00640 1 -- 172.21.15.106:0/976574644 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4009820 con 0x7f45bc15cfd0 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 -- 172.21.15.106:0/976574644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cfd0 msgr2=0x7f45bc073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 --2- 172.21.15.106:0/976574644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cfd0 0x7f45bc073830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0741f0 tx=0x7f45b402c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 -- 172.21.15.106:0/976574644 shutdown_connections 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 --2- 172.21.15.106:0/976574644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cfd0 0x7f45bc073830 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 --2- 172.21.15.106:0/976574644 >> v2:172.21.15.204:3300/0 conn(0x7f45bc06c770 0x7f45bc06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 -- 172.21.15.106:0/976574644 >> 172.21.15.106:0/976574644 conn(0x7f45bc0b2d50 msgr2=0x7f45bc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 -- 172.21.15.106:0/976574644 shutdown_connections 2024-04-26T21:47:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.747+0000 7f45c4600640 1 -- 172.21.15.106:0/976574644 wait complete. 2024-04-26T21:47:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.749+0000 7f45c4600640 1 Processor -- start 2024-04-26T21:47:18.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.749+0000 7f45c4600640 1 -- start start 2024-04-26T21:47:18.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc06c770 0x7f45bc15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 0x7f45bc073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc15c600 con 0x7f45bc15cb70 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0af0f0 con 0x7f45bc06c770 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc15c780 con 0x7f45bc15cfd0 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53488/0 (socket says 172.21.15.106:53488) 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c1200640 1 -- 172.21.15.106:0/34164809 learned_addr learned my addr 172.21.15.106:0/34164809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.750+0000 7f45c1c00640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc06c770 0x7f45bc15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c2600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 0x7f45bc073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 -- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 msgr2=0x7f45bc073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 0x7f45bc073060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 -- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc06c770 msgr2=0x7f45bc15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc06c770 0x7f45bc15ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 -- 172.21.15.106:0/34164809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc09e850 con 0x7f45bc15cb70 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c2600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 0x7f45bc073060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c1200640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c460 tx=0x7f45a800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011790 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a8017070 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011bb0 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc15f260 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.751+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc0ca330 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.752+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc069540 con 0x7f45bc15cb70 2024-04-26T21:47:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.753+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45a801c460 con 0x7f45bc15cb70 2024-04-26T21:47:18.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.754+0000 7f45b9600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.754+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f45a808b9b0 con 0x7f45bc15cb70 2024-04-26T21:47:18.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.755+0000 7f45c1c00640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.755+0000 7f45c1c00640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 0x7f459c06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45b000a990 tx=0x7f45b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:18.759+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45a8059de0 con 0x7f45bc15cb70 2024-04-26T21:47:19.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.002+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45bc0cd430 con 0x7f45bc15cb70 2024-04-26T21:47:19.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.003+0000 7f45b9600640 1 -- 172.21.15.106:0/34164809 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f45a80596e0 con 0x7f45bc15cb70 2024-04-26T21:47:19.003 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:19.004 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:19.004 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 msgr2=0x7f459c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 0x7f459c06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45b000a990 tx=0x7f45b0008040 comp rx=0 tx=0).stop 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 msgr2=0x7f45bc14e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c460 tx=0x7f45a800c920 comp rx=0 tx=0).stop 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 shutdown_connections 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc15cfd0 0x7f45bc073060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45bc15cb70 0x7f45bc14e730 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 --2- 172.21.15.106:0/34164809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc06c770 0x7f45bc15ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 >> 172.21.15.106:0/34164809 conn(0x7f45bc0b2d50 msgr2=0x7f45bc15dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 shutdown_connections 2024-04-26T21:47:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.006+0000 7f45c4600640 1 -- 172.21.15.106:0/34164809 wait complete. 2024-04-26T21:47:19.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:19.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.226+0000 7f053da00640 1 Processor -- start 2024-04-26T21:47:19.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.226+0000 7f053da00640 1 -- start start 2024-04-26T21:47:19.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.227+0000 7f053da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f05380add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.228+0000 7f053da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05380cd650 0x7f053806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.228+0000 7f053da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05380af9a0 con 0x7f05380b4f50 2024-04-26T21:47:19.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.228+0000 7f053da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f05380ae2c0 con 0x7f05380cd650 2024-04-26T21:47:19.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.228+0000 7f053da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f05380b05b0 con 0x7f053815cdf0 2024-04-26T21:47:19.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.228+0000 7f0536a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f05380add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.229+0000 7f0536000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05380cd650 0x7f053806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.229+0000 7f0537400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f05380b4f50 0x7f0538072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:52164/0 (socket says 172.21.15.106:52164) 2024-04-26T21:47:19.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.229+0000 7f0537400640 1 -- 172.21.15.106:0/1684204170 learned_addr learned my addr 172.21.15.106:0/1684204170 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:19.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.229+0000 7f0536000640 1 -- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 msgr2=0x7f05380add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:19.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.229+0000 7f0536000640 1 --2- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f05380add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.230+0000 7f0536000640 1 -- 172.21.15.106:0/1684204170 >> v1:172.21.15.204:6789/0 conn(0x7f05380b4f50 legacy=0x7f0538072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.230+0000 7f0536000640 1 -- 172.21.15.106:0/1684204170 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05380afc80 con 0x7f05380cd650 2024-04-26T21:47:19.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.231+0000 7f0536000640 1 --2- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380cd650 0x7f053806c770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6061bc88dfbc0410 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.232+0000 7f0535600640 1 -- 172.21.15.106:0/1684204170 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524037070 con 0x7f05380cd650 2024-04-26T21:47:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.232+0000 7f0535600640 1 -- 172.21.15.106:0/1684204170 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0524032440 con 0x7f05380cd650 2024-04-26T21:47:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.232+0000 7f0535600640 1 -- 172.21.15.106:0/1684204170 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524032810 con 0x7f05380cd650 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 -- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380cd650 msgr2=0x7f053806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 --2- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380cd650 0x7f053806c770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402a180 comp rx=0 tx=0).stop 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 -- 172.21.15.106:0/1684204170 shutdown_connections 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 --2- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380cd650 0x7f053806c770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 --2- 172.21.15.106:0/1684204170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f05380add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 -- 172.21.15.106:0/1684204170 >> 172.21.15.106:0/1684204170 conn(0x7f05380b2d50 msgr2=0x7f05380b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 -- 172.21.15.106:0/1684204170 shutdown_connections 2024-04-26T21:47:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.233+0000 7f053da00640 1 -- 172.21.15.106:0/1684204170 wait complete. 2024-04-26T21:47:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.235+0000 7f053da00640 1 Processor -- start 2024-04-26T21:47:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.235+0000 7f053da00640 1 -- start start 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.235+0000 7f053da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380b4f50 0x7f053814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f053da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f053da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f0538152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f053da00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f053815d1d0 con 0x7f05380cd650 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f053da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05380b0240 con 0x7f05380b4f50 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f053da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f053815d350 con 0x7f053815cdf0 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53494/0 (socket says 172.21.15.106:53494) 2024-04-26T21:47:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 -- 172.21.15.106:0/2011320317 learned_addr learned my addr 172.21.15.106:0/2011320317 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536a00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380b4f50 0x7f053814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0537400640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f0538152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 -- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 msgr2=0x7f0538152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f0538152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 -- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380b4f50 msgr2=0x7f053814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380b4f50 0x7f053814d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0536000640 1 -- 172.21.15.106:0/2011320317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0538152750 con 0x7f05380cd650 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.236+0000 7f0537400640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f0538152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f0536000640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0524035040 tx=0x7f05240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524037040 con 0x7f05380cd650 2024-04-26T21:47:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0524030500 con 0x7f05380cd650 2024-04-26T21:47:19.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524030910 con 0x7f05380cd650 2024-04-26T21:47:19.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0538156a40 con 0x7f05380cd650 2024-04-26T21:47:19.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.237+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0538157000 con 0x7f05380cd650 2024-04-26T21:47:19.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.238+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0538069540 con 0x7f05380cd650 2024-04-26T21:47:19.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.239+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0524044020 con 0x7f05380cd650 2024-04-26T21:47:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.240+0000 7f052ea00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 0x7f051806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.240+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f05240abec0 con 0x7f05380cd650 2024-04-26T21:47:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.241+0000 7f0536a00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 0x7f051806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.241+0000 7f0536a00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 0x7f051806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0528009ad0 tx=0x7f0528008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.245+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f052407a250 con 0x7f05380cd650 2024-04-26T21:47:19.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.446+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0538072760 con 0x7f05380cd650 2024-04-26T21:47:19.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.448+0000 7f052ea00640 1 -- 172.21.15.106:0/2011320317 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f0524079b50 con 0x7f05380cd650 2024-04-26T21:47:19.448 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:19.449 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 msgr2=0x7f051806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 0x7f051806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0528009ad0 tx=0x7f0528008040 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 msgr2=0x7f0538156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0524035040 tx=0x7f05240046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 shutdown_connections 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f051806c8f0 0x7f051806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f053815cdf0 0x7f0538152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05380cd650 0x7f0538156310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 --2- 172.21.15.106:0/2011320317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05380b4f50 0x7f053814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 >> 172.21.15.106:0/2011320317 conn(0x7f05380b2d50 msgr2=0x7f05380b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 shutdown_connections 2024-04-26T21:47:19.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.451+0000 7f053da00640 1 -- 172.21.15.106:0/2011320317 wait complete. 2024-04-26T21:47:19.469 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:47:19.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.669+0000 7fec87c00640 1 Processor -- start 2024-04-26T21:47:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.670+0000 7fec87c00640 1 -- start start 2024-04-26T21:47:19.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec87c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec8009ba50 0x7fec8009be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec87c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec8006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec87c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8009c370 con 0x7fec8006f230 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec87c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fec80070420 con 0x7fec8009ba50 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec87c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fec8006e790 con 0x7fec800b58a0 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec85200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec8009ba50 0x7fec8009be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec85200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec8009ba50 0x7fec8009be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53498/0 (socket says 172.21.15.106:53498) 2024-04-26T21:47:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.671+0000 7fec85200640 1 -- 172.21.15.106:0/3614212055 learned_addr learned my addr 172.21.15.106:0/3614212055 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:19.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.672+0000 7fec85200640 1 -- 172.21.15.106:0/3614212055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 msgr2=0x7fec8006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:19.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.672+0000 7fec85200640 1 --2- 172.21.15.106:0/3614212055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec8006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.672+0000 7fec85200640 1 -- 172.21.15.106:0/3614212055 >> v1:172.21.15.204:6789/0 conn(0x7fec8006f230 legacy=0x7fec8006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:19.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.672+0000 7fec85200640 1 -- 172.21.15.106:0/3614212055 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec80070670 con 0x7fec8009ba50 2024-04-26T21:47:19.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.673+0000 7fec85200640 1 --2- 172.21.15.106:0/3614212055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec8009be30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fec70009870 tx=0x7fec7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed9bff97831f1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.673+0000 7fec7f400640 1 -- 172.21.15.106:0/3614212055 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70037070 con 0x7fec8009ba50 2024-04-26T21:47:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.673+0000 7fec7f400640 1 -- 172.21.15.106:0/3614212055 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70032440 con 0x7fec8009ba50 2024-04-26T21:47:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.673+0000 7fec7f400640 1 -- 172.21.15.106:0/3614212055 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70032810 con 0x7fec8009ba50 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 -- 172.21.15.106:0/3614212055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 msgr2=0x7fec8009be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 --2- 172.21.15.106:0/3614212055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec8009be30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fec70009870 tx=0x7fec7002a180 comp rx=0 tx=0).stop 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 -- 172.21.15.106:0/3614212055 shutdown_connections 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 --2- 172.21.15.106:0/3614212055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec8006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 --2- 172.21.15.106:0/3614212055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec8009be30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 -- 172.21.15.106:0/3614212055 >> 172.21.15.106:0/3614212055 conn(0x7fec800b4b70 msgr2=0x7fec800b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 -- 172.21.15.106:0/3614212055 shutdown_connections 2024-04-26T21:47:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.674+0000 7fec87c00640 1 -- 172.21.15.106:0/3614212055 wait complete. 2024-04-26T21:47:19.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.676+0000 7fec87c00640 1 Processor -- start 2024-04-26T21:47:19.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.676+0000 7fec87c00640 1 -- start start 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8006f230 0x7fec800d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec801e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec8006ecc0 con 0x7fec8009ba50 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec8006e9e0 con 0x7fec8006f230 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec87c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fec800cf440 con 0x7fec800b58a0 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.677+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53508/0 (socket says 172.21.15.106:53508) 2024-04-26T21:47:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 -- 172.21.15.106:0/594910662 learned_addr learned my addr 172.21.15.106:0/594910662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec85200640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8006f230 0x7fec800d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec85c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec801e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 -- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 msgr2=0x7fec801e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec801e8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 -- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8006f230 msgr2=0x7fec800d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8006f230 0x7fec800d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 -- 172.21.15.106:0/594910662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec800cc6c0 con 0x7fec8009ba50 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7fe00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ba80 tx=0x7fec6c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.678+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c019070 con 0x7fec8009ba50 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.679+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6c00cc20 con 0x7fec8009ba50 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.679+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c014440 con 0x7fec8009ba50 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.679+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec800caf20 con 0x7fec8009ba50 2024-04-26T21:47:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.679+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec800cc960 con 0x7fec8009ba50 2024-04-26T21:47:19.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.680+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec80072550 con 0x7fec8009ba50 2024-04-26T21:47:19.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.681+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec6c01c070 con 0x7fec8009ba50 2024-04-26T21:47:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.682+0000 7fec7cc00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 0x7fec6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.682+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fec6c08b7a0 con 0x7fec8009ba50 2024-04-26T21:47:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.682+0000 7fec85200640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 0x7fec6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.683+0000 7fec85200640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 0x7fec6006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec70034000 tx=0x7fec7002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.686+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec6c059580 con 0x7fec8009ba50 2024-04-26T21:47:19.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.882+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec80069540 con 0x7fec6006c8f0 2024-04-26T21:47:19.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.902+0000 7fec7cc00640 1 -- 172.21.15.106:0/594910662 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7fec80069540 con 0x7fec6006c8f0 2024-04-26T21:47:19.906 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:19.908 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:47:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 msgr2=0x7fec6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 0x7fec6006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec70034000 tx=0x7fec7002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 msgr2=0x7fec801e16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ba80 tx=0x7fec6c00bf40 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 shutdown_connections 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec6006c8f0 0x7fec6006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec800b58a0 0x7fec801e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec8009ba50 0x7fec801e16f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 --2- 172.21.15.106:0/594910662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8006f230 0x7fec800d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 >> 172.21.15.106:0/594910662 conn(0x7fec800b4b70 msgr2=0x7fec801e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 shutdown_connections 2024-04-26T21:47:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:19.909+0000 7fec87c00640 1 -- 172.21.15.106:0/594910662 wait complete. 2024-04-26T21:47:19.927 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T21:47:19.113548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220020,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754700,"statfs":{"total":773094113280,"available":772868812800,"internally_reserved":0,"allocated":3936256,"data_stored":2230199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.676889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469233+0000","last_change":"2024-04-26T21:47:14.469233+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469233+0000","last_undegraded":"2024-04-26T21:47:14.469233+0000","last_fullsized":"2024-04-26T21:47:14.469233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.473349+0000","last_change":"2024-04-26T21:47:14.473349+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.473349+0000","last_undegraded":"2024-04-26T21:47:14.473349+0000","last_fullsized":"2024-04-26T21:47:14.473349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466695+0000","last_change":"2024-04-26T21:47:14.466695+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466695+0000","last_undegraded":"2024-04-26T21:47:14.466695+0000","last_fullsized":"2024-04-26T21:47:14.466695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463852+0000","last_change":"2024-04-26T21:47:14.463852+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463852+0000","last_undegraded":"2024-04-26T21:47:14.463852+0000","last_fullsized":"2024-04-26T21:47:14.463852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466007+0000","last_change":"2024-04-26T21:47:14.466007+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466007+0000","last_undegraded":"2024-04-26T21:47:14.466007+0000","last_fullsized":"2024-04-26T21:47:14.466007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.471474+0000","last_change":"2024-04-26T21:47:14.471474+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.471474+0000","last_undegraded":"2024-04-26T21:47:14.471474+0000","last_fullsized":"2024-04-26T21:47:14.471474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452768+0000","last_change":"2024-04-26T21:47:14.452768+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452768+0000","last_undegraded":"2024-04-26T21:47:14.452768+0000","last_fullsized":"2024-04-26T21:47:14.452768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457237+0000","last_change":"2024-04-26T21:47:14.457237+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457237+0000","last_undegraded":"2024-04-26T21:47:14.457237+0000","last_fullsized":"2024-04-26T21:47:14.457237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453766+0000","last_change":"2024-04-26T21:47:14.453766+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453766+0000","last_undegraded":"2024-04-26T21:47:14.453766+0000","last_fullsized":"2024-04-26T21:47:14.453766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462059+0000","last_change":"2024-04-26T21:47:14.462059+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462059+0000","last_undegraded":"2024-04-26T21:47:14.462059+0000","last_fullsized":"2024-04-26T21:47:14.462059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453846+0000","last_change":"2024-04-26T21:47:14.453846+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453846+0000","last_undegraded":"2024-04-26T21:47:14.453846+0000","last_fullsized":"2024-04-26T21:47:14.453846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465401+0000","last_change":"2024-04-26T21:47:14.465401+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465401+0000","last_undegraded":"2024-04-26T21:47:14.465401+0000","last_fullsized":"2024-04-26T21:47:14.465401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459374+0000","last_change":"2024-04-26T21:47:14.459374+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459374+0000","last_undegraded":"2024-04-26T21:47:14.459374+0000","last_fullsized":"2024-04-26T21:47:14.459374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470943+0000","last_change":"2024-04-26T21:47:14.470943+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470943+0000","last_undegraded":"2024-04-26T21:47:14.470943+0000","last_fullsized":"2024-04-26T21:47:14.470943+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470934+0000","last_change":"2024-04-26T21:47:14.470934+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470934+0000","last_undegraded":"2024-04-26T21:47:14.470934+0000","last_fullsized":"2024-04-26T21:47:14.470934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449141+0000","last_change":"2024-04-26T21:47:14.449141+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449141+0000","last_undegraded":"2024-04-26T21:47:14.449141+0000","last_fullsized":"2024-04-26T21:47:14.449141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450358+0000","last_change":"2024-04-26T21:47:14.450358+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450358+0000","last_undegraded":"2024-04-26T21:47:14.450358+0000","last_fullsized":"2024-04-26T21:47:14.450358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470776+0000","last_change":"2024-04-26T21:47:14.470776+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470776+0000","last_undegraded":"2024-04-26T21:47:14.470776+0000","last_fullsized":"2024-04-26T21:47:14.470776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469285+0000","last_change":"2024-04-26T21:47:14.469285+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469285+0000","last_undegraded":"2024-04-26T21:47:14.469285+0000","last_fullsized":"2024-04-26T21:47:14.469285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457654+0000","last_change":"2024-04-26T21:47:14.457654+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457654+0000","last_undegraded":"2024-04-26T21:47:14.457654+0000","last_fullsized":"2024-04-26T21:47:14.457654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457377+0000","last_change":"2024-04-26T21:47:14.457377+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457377+0000","last_undegraded":"2024-04-26T21:47:14.457377+0000","last_fullsized":"2024-04-26T21:47:14.457377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452220+0000","last_change":"2024-04-26T21:47:14.452220+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452220+0000","last_undegraded":"2024-04-26T21:47:14.452220+0000","last_fullsized":"2024-04-26T21:47:14.452220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462364+0000","last_change":"2024-04-26T21:47:14.462364+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462364+0000","last_undegraded":"2024-04-26T21:47:14.462364+0000","last_fullsized":"2024-04-26T21:47:14.462364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469923+0000","last_change":"2024-04-26T21:47:14.469923+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469923+0000","last_undegraded":"2024-04-26T21:47:14.469923+0000","last_fullsized":"2024-04-26T21:47:14.469923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.460700+0000","last_change":"2024-04-26T21:47:14.460700+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.460700+0000","last_undegraded":"2024-04-26T21:47:14.460700+0000","last_fullsized":"2024-04-26T21:47:14.460700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468882+0000","last_change":"2024-04-26T21:47:14.468882+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468882+0000","last_undegraded":"2024-04-26T21:47:14.468882+0000","last_fullsized":"2024-04-26T21:47:14.468882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465607+0000","last_change":"2024-04-26T21:47:14.465607+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465607+0000","last_undegraded":"2024-04-26T21:47:14.465607+0000","last_fullsized":"2024-04-26T21:47:14.465607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463389+0000","last_change":"2024-04-26T21:47:14.463389+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463389+0000","last_undegraded":"2024-04-26T21:47:14.463389+0000","last_fullsized":"2024-04-26T21:47:14.463389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456256+0000","last_change":"2024-04-26T21:47:14.456256+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456256+0000","last_undegraded":"2024-04-26T21:47:14.456256+0000","last_fullsized":"2024-04-26T21:47:14.456256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450566+0000","last_change":"2024-04-26T21:47:14.450566+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450566+0000","last_undegraded":"2024-04-26T21:47:14.450566+0000","last_fullsized":"2024-04-26T21:47:14.450566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468593+0000","last_change":"2024-04-26T21:47:14.468593+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468593+0000","last_undegraded":"2024-04-26T21:47:14.468593+0000","last_fullsized":"2024-04-26T21:47:14.468593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456198+0000","last_change":"2024-04-26T21:47:14.456198+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456198+0000","last_undegraded":"2024-04-26T21:47:14.456198+0000","last_fullsized":"2024-04-26T21:47:14.456198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470298+0000","last_change":"2024-04-26T21:47:14.470298+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470298+0000","last_undegraded":"2024-04-26T21:47:14.470298+0000","last_fullsized":"2024-04-26T21:47:14.470298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488061+0000","last_change":"2024-04-26T21:47:14.483471+0000","last_active":"2024-04-26T21:47:14.488061+0000","last_peered":"2024-04-26T21:47:14.488061+0000","last_clean":"2024-04-26T21:47:14.488061+0000","last_became_active":"2024-04-26T21:47:14.483132+0000","last_became_peered":"2024-04-26T21:47:14.483132+0000","last_unstale":"2024-04-26T21:47:14.488061+0000","last_undegraded":"2024-04-26T21:47:14.488061+0000","last_fullsized":"2024-04-26T21:47:14.488061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468665+0000","last_change":"2024-04-26T21:47:14.468665+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468665+0000","last_undegraded":"2024-04-26T21:47:14.468665+0000","last_fullsized":"2024-04-26T21:47:14.468665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486611+0000","last_change":"2024-04-26T21:47:14.477793+0000","last_active":"2024-04-26T21:47:14.486611+0000","last_peered":"2024-04-26T21:47:14.486611+0000","last_clean":"2024-04-26T21:47:14.486611+0000","last_became_active":"2024-04-26T21:47:14.477460+0000","last_became_peered":"2024-04-26T21:47:14.477460+0000","last_unstale":"2024-04-26T21:47:14.486611+0000","last_undegraded":"2024-04-26T21:47:14.486611+0000","last_fullsized":"2024-04-26T21:47:14.486611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453644+0000","last_change":"2024-04-26T21:47:14.453644+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453644+0000","last_undegraded":"2024-04-26T21:47:14.453644+0000","last_fullsized":"2024-04-26T21:47:14.453644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490372+0000","last_change":"2024-04-26T21:47:14.484669+0000","last_active":"2024-04-26T21:47:14.490372+0000","last_peered":"2024-04-26T21:47:14.490372+0000","last_clean":"2024-04-26T21:47:14.490372+0000","last_became_active":"2024-04-26T21:47:14.483933+0000","last_became_peered":"2024-04-26T21:47:14.483933+0000","last_unstale":"2024-04-26T21:47:14.490372+0000","last_undegraded":"2024-04-26T21:47:14.490372+0000","last_fullsized":"2024-04-26T21:47:14.490372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463630+0000","last_change":"2024-04-26T21:47:14.463630+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463630+0000","last_undegraded":"2024-04-26T21:47:14.463630+0000","last_fullsized":"2024-04-26T21:47:14.463630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.469248+0000","last_change":"2024-04-26T21:47:14.462571+0000","last_active":"2024-04-26T21:47:14.469248+0000","last_peered":"2024-04-26T21:47:14.469248+0000","last_clean":"2024-04-26T21:47:14.469248+0000","last_became_active":"2024-04-26T21:47:14.461470+0000","last_became_peered":"2024-04-26T21:47:14.461470+0000","last_unstale":"2024-04-26T21:47:14.469248+0000","last_undegraded":"2024-04-26T21:47:14.469248+0000","last_fullsized":"2024-04-26T21:47:14.469248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449275+0000","last_change":"2024-04-26T21:47:14.449275+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449275+0000","last_undegraded":"2024-04-26T21:47:14.449275+0000","last_fullsized":"2024-04-26T21:47:14.449275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490646+0000","last_change":"2024-04-26T21:47:14.485718+0000","last_active":"2024-04-26T21:47:14.490646+0000","last_peered":"2024-04-26T21:47:14.490646+0000","last_clean":"2024-04-26T21:47:14.490646+0000","last_became_active":"2024-04-26T21:47:14.485458+0000","last_became_peered":"2024-04-26T21:47:14.485458+0000","last_unstale":"2024-04-26T21:47:14.490646+0000","last_undegraded":"2024-04-26T21:47:14.490646+0000","last_fullsized":"2024-04-26T21:47:14.490646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:12.839326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463772+0000","last_change":"2024-04-26T21:47:14.463772+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463772+0000","last_undegraded":"2024-04-26T21:47:14.463772+0000","last_fullsized":"2024-04-26T21:47:14.463772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488020+0000","last_change":"2024-04-26T21:47:14.484537+0000","last_active":"2024-04-26T21:47:14.488020+0000","last_peered":"2024-04-26T21:47:14.488020+0000","last_clean":"2024-04-26T21:47:14.488020+0000","last_became_active":"2024-04-26T21:47:14.484218+0000","last_became_peered":"2024-04-26T21:47:14.484218+0000","last_unstale":"2024-04-26T21:47:14.488020+0000","last_undegraded":"2024-04-26T21:47:14.488020+0000","last_fullsized":"2024-04-26T21:47:14.488020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:23.713847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452619+0000","last_change":"2024-04-26T21:47:14.452619+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452619+0000","last_undegraded":"2024-04-26T21:47:14.452619+0000","last_fullsized":"2024-04-26T21:47:14.452619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490039+0000","last_change":"2024-04-26T21:47:14.483496+0000","last_active":"2024-04-26T21:47:14.490039+0000","last_peered":"2024-04-26T21:47:14.490039+0000","last_clean":"2024-04-26T21:47:14.490039+0000","last_became_active":"2024-04-26T21:47:14.483201+0000","last_became_peered":"2024-04-26T21:47:14.483201+0000","last_unstale":"2024-04-26T21:47:14.490039+0000","last_undegraded":"2024-04-26T21:47:14.490039+0000","last_fullsized":"2024-04-26T21:47:14.490039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463534+0000","last_change":"2024-04-26T21:47:14.463534+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463534+0000","last_undegraded":"2024-04-26T21:47:14.463534+0000","last_fullsized":"2024-04-26T21:47:14.463534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482982+0000","last_change":"2024-04-26T21:47:14.480135+0000","last_active":"2024-04-26T21:47:14.482982+0000","last_peered":"2024-04-26T21:47:14.482982+0000","last_clean":"2024-04-26T21:47:14.482982+0000","last_became_active":"2024-04-26T21:47:14.479762+0000","last_became_peered":"2024-04-26T21:47:14.479762+0000","last_unstale":"2024-04-26T21:47:14.482982+0000","last_undegraded":"2024-04-26T21:47:14.482982+0000","last_fullsized":"2024-04-26T21:47:14.482982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:37.394548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463597+0000","last_change":"2024-04-26T21:47:14.463597+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463597+0000","last_undegraded":"2024-04-26T21:47:14.463597+0000","last_fullsized":"2024-04-26T21:47:14.463597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490144+0000","last_change":"2024-04-26T21:47:14.484137+0000","last_active":"2024-04-26T21:47:14.490144+0000","last_peered":"2024-04-26T21:47:14.490144+0000","last_clean":"2024-04-26T21:47:14.490144+0000","last_became_active":"2024-04-26T21:47:14.483547+0000","last_became_peered":"2024-04-26T21:47:14.483547+0000","last_unstale":"2024-04-26T21:47:14.490144+0000","last_undegraded":"2024-04-26T21:47:14.490144+0000","last_fullsized":"2024-04-26T21:47:14.490144+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.476712+0000","last_change":"2024-04-26T21:47:14.476712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.476712+0000","last_undegraded":"2024-04-26T21:47:14.476712+0000","last_fullsized":"2024-04-26T21:47:14.476712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489763+0000","last_change":"2024-04-26T21:47:14.485542+0000","last_active":"2024-04-26T21:47:14.489763+0000","last_peered":"2024-04-26T21:47:14.489763+0000","last_clean":"2024-04-26T21:47:14.489763+0000","last_became_active":"2024-04-26T21:47:14.485231+0000","last_became_peered":"2024-04-26T21:47:14.485231+0000","last_unstale":"2024-04-26T21:47:14.489763+0000","last_undegraded":"2024-04-26T21:47:14.489763+0000","last_fullsized":"2024-04-26T21:47:14.489763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.475921+0000","last_change":"2024-04-26T21:47:14.475921+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.475921+0000","last_undegraded":"2024-04-26T21:47:14.475921+0000","last_fullsized":"2024-04-26T21:47:14.475921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.480096+0000","last_change":"2024-04-26T21:47:14.472755+0000","last_active":"2024-04-26T21:47:14.480096+0000","last_peered":"2024-04-26T21:47:14.480096+0000","last_clean":"2024-04-26T21:47:14.480096+0000","last_became_active":"2024-04-26T21:47:14.472547+0000","last_became_peered":"2024-04-26T21:47:14.472547+0000","last_unstale":"2024-04-26T21:47:14.480096+0000","last_undegraded":"2024-04-26T21:47:14.480096+0000","last_fullsized":"2024-04-26T21:47:14.480096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467306+0000","last_change":"2024-04-26T21:47:14.467306+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467306+0000","last_undegraded":"2024-04-26T21:47:14.467306+0000","last_fullsized":"2024-04-26T21:47:14.467306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493863+0000","last_change":"2024-04-26T21:47:14.487095+0000","last_active":"2024-04-26T21:47:14.493863+0000","last_peered":"2024-04-26T21:47:14.493863+0000","last_clean":"2024-04-26T21:47:14.493863+0000","last_became_active":"2024-04-26T21:47:14.486802+0000","last_became_peered":"2024-04-26T21:47:14.486802+0000","last_unstale":"2024-04-26T21:47:14.493863+0000","last_undegraded":"2024-04-26T21:47:14.493863+0000","last_fullsized":"2024-04-26T21:47:14.493863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.461282+0000","last_change":"2024-04-26T21:47:14.461282+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.461282+0000","last_undegraded":"2024-04-26T21:47:14.461282+0000","last_fullsized":"2024-04-26T21:47:14.461282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481227+0000","last_change":"2024-04-26T21:47:14.478529+0000","last_active":"2024-04-26T21:47:14.481227+0000","last_peered":"2024-04-26T21:47:14.481227+0000","last_clean":"2024-04-26T21:47:14.481227+0000","last_became_active":"2024-04-26T21:47:14.478281+0000","last_became_peered":"2024-04-26T21:47:14.478281+0000","last_unstale":"2024-04-26T21:47:14.481227+0000","last_undegraded":"2024-04-26T21:47:14.481227+0000","last_fullsized":"2024-04-26T21:47:14.481227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:02.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465187+0000","last_change":"2024-04-26T21:47:14.465187+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465187+0000","last_undegraded":"2024-04-26T21:47:14.465187+0000","last_fullsized":"2024-04-26T21:47:14.465187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481931+0000","last_change":"2024-04-26T21:47:14.480124+0000","last_active":"2024-04-26T21:47:14.481931+0000","last_peered":"2024-04-26T21:47:14.481931+0000","last_clean":"2024-04-26T21:47:14.481931+0000","last_became_active":"2024-04-26T21:47:14.479846+0000","last_became_peered":"2024-04-26T21:47:14.479846+0000","last_unstale":"2024-04-26T21:47:14.481931+0000","last_undegraded":"2024-04-26T21:47:14.481931+0000","last_fullsized":"2024-04-26T21:47:14.481931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.455199+0000","last_change":"2024-04-26T21:47:14.455199+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.455199+0000","last_undegraded":"2024-04-26T21:47:14.455199+0000","last_fullsized":"2024-04-26T21:47:14.455199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490380+0000","last_change":"2024-04-26T21:47:14.485004+0000","last_active":"2024-04-26T21:47:14.490380+0000","last_peered":"2024-04-26T21:47:14.490380+0000","last_clean":"2024-04-26T21:47:14.490380+0000","last_became_active":"2024-04-26T21:47:14.483355+0000","last_became_peered":"2024-04-26T21:47:14.483355+0000","last_unstale":"2024-04-26T21:47:14.490380+0000","last_undegraded":"2024-04-26T21:47:14.490380+0000","last_fullsized":"2024-04-26T21:47:14.490380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467945+0000","last_change":"2024-04-26T21:47:14.467945+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467945+0000","last_undegraded":"2024-04-26T21:47:14.467945+0000","last_fullsized":"2024-04-26T21:47:14.467945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487389+0000","last_change":"2024-04-26T21:47:14.482285+0000","last_active":"2024-04-26T21:47:14.487389+0000","last_peered":"2024-04-26T21:47:14.487389+0000","last_clean":"2024-04-26T21:47:14.487389+0000","last_became_active":"2024-04-26T21:47:14.481952+0000","last_became_peered":"2024-04-26T21:47:14.481952+0000","last_unstale":"2024-04-26T21:47:14.487389+0000","last_undegraded":"2024-04-26T21:47:14.487389+0000","last_fullsized":"2024-04-26T21:47:14.487389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456504+0000","last_change":"2024-04-26T21:47:14.456504+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456504+0000","last_undegraded":"2024-04-26T21:47:14.456504+0000","last_fullsized":"2024-04-26T21:47:14.456504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493983+0000","last_change":"2024-04-26T21:47:14.485819+0000","last_active":"2024-04-26T21:47:14.493983+0000","last_peered":"2024-04-26T21:47:14.493983+0000","last_clean":"2024-04-26T21:47:14.493983+0000","last_became_active":"2024-04-26T21:47:14.485435+0000","last_became_peered":"2024-04-26T21:47:14.485435+0000","last_unstale":"2024-04-26T21:47:14.493983+0000","last_undegraded":"2024-04-26T21:47:14.493983+0000","last_fullsized":"2024-04-26T21:47:14.493983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465748+0000","last_change":"2024-04-26T21:47:14.465748+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465748+0000","last_undegraded":"2024-04-26T21:47:14.465748+0000","last_fullsized":"2024-04-26T21:47:14.465748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485819+0000","last_change":"2024-04-26T21:47:14.481662+0000","last_active":"2024-04-26T21:47:14.485819+0000","last_peered":"2024-04-26T21:47:14.485819+0000","last_clean":"2024-04-26T21:47:14.485819+0000","last_became_active":"2024-04-26T21:47:14.480582+0000","last_became_peered":"2024-04-26T21:47:14.480582+0000","last_unstale":"2024-04-26T21:47:14.485819+0000","last_undegraded":"2024-04-26T21:47:14.485819+0000","last_fullsized":"2024-04-26T21:47:14.485819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:00.441765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462995+0000","last_change":"2024-04-26T21:47:14.462995+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462995+0000","last_undegraded":"2024-04-26T21:47:14.462995+0000","last_fullsized":"2024-04-26T21:47:14.462995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478547+0000","last_change":"2024-04-26T21:47:14.475423+0000","last_active":"2024-04-26T21:47:14.478547+0000","last_peered":"2024-04-26T21:47:14.478547+0000","last_clean":"2024-04-26T21:47:14.478547+0000","last_became_active":"2024-04-26T21:47:14.475083+0000","last_became_peered":"2024-04-26T21:47:14.475083+0000","last_unstale":"2024-04-26T21:47:14.478547+0000","last_undegraded":"2024-04-26T21:47:14.478547+0000","last_fullsized":"2024-04-26T21:47:14.478547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459712+0000","last_change":"2024-04-26T21:47:14.459712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459712+0000","last_undegraded":"2024-04-26T21:47:14.459712+0000","last_fullsized":"2024-04-26T21:47:14.459712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490682+0000","last_change":"2024-04-26T21:47:14.487648+0000","last_active":"2024-04-26T21:47:14.490682+0000","last_peered":"2024-04-26T21:47:14.490682+0000","last_clean":"2024-04-26T21:47:14.490682+0000","last_became_active":"2024-04-26T21:47:14.487356+0000","last_became_peered":"2024-04-26T21:47:14.487356+0000","last_unstale":"2024-04-26T21:47:14.490682+0000","last_undegraded":"2024-04-26T21:47:14.490682+0000","last_fullsized":"2024-04-26T21:47:14.490682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:02.916803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458838+0000","last_change":"2024-04-26T21:47:14.458838+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458838+0000","last_undegraded":"2024-04-26T21:47:14.458838+0000","last_fullsized":"2024-04-26T21:47:14.458838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486377+0000","last_change":"2024-04-26T21:47:14.478983+0000","last_active":"2024-04-26T21:47:14.486377+0000","last_peered":"2024-04-26T21:47:14.486377+0000","last_clean":"2024-04-26T21:47:14.486377+0000","last_became_active":"2024-04-26T21:47:14.478698+0000","last_became_peered":"2024-04-26T21:47:14.478698+0000","last_unstale":"2024-04-26T21:47:14.486377+0000","last_undegraded":"2024-04-26T21:47:14.486377+0000","last_fullsized":"2024-04-26T21:47:14.486377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458086+0000","last_change":"2024-04-26T21:47:14.458086+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458086+0000","last_undegraded":"2024-04-26T21:47:14.458086+0000","last_fullsized":"2024-04-26T21:47:14.458086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487855+0000","last_change":"2024-04-26T21:47:14.482597+0000","last_active":"2024-04-26T21:47:14.487855+0000","last_peered":"2024-04-26T21:47:14.487855+0000","last_clean":"2024-04-26T21:47:14.487855+0000","last_became_active":"2024-04-26T21:47:14.482299+0000","last_became_peered":"2024-04-26T21:47:14.482299+0000","last_unstale":"2024-04-26T21:47:14.487855+0000","last_undegraded":"2024-04-26T21:47:14.487855+0000","last_fullsized":"2024-04-26T21:47:14.487855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:20.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457559+0000","last_change":"2024-04-26T21:47:14.457559+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457559+0000","last_undegraded":"2024-04-26T21:47:14.457559+0000","last_fullsized":"2024-04-26T21:47:14.457559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478807+0000","last_change":"2024-04-26T21:47:14.475056+0000","last_active":"2024-04-26T21:47:14.478807+0000","last_peered":"2024-04-26T21:47:14.478807+0000","last_clean":"2024-04-26T21:47:14.478807+0000","last_became_active":"2024-04-26T21:47:14.474735+0000","last_became_peered":"2024-04-26T21:47:14.474735+0000","last_unstale":"2024-04-26T21:47:14.478807+0000","last_undegraded":"2024-04-26T21:47:14.478807+0000","last_fullsized":"2024-04-26T21:47:14.478807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:06.758168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453724+0000","last_change":"2024-04-26T21:47:14.453724+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453724+0000","last_undegraded":"2024-04-26T21:47:14.453724+0000","last_fullsized":"2024-04-26T21:47:14.453724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482671+0000","last_change":"2024-04-26T21:47:14.478761+0000","last_active":"2024-04-26T21:47:14.482671+0000","last_peered":"2024-04-26T21:47:14.482671+0000","last_clean":"2024-04-26T21:47:14.482671+0000","last_became_active":"2024-04-26T21:47:14.477387+0000","last_became_peered":"2024-04-26T21:47:14.477387+0000","last_unstale":"2024-04-26T21:47:14.482671+0000","last_undegraded":"2024-04-26T21:47:14.482671+0000","last_fullsized":"2024-04-26T21:47:14.482671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457583+0000","last_change":"2024-04-26T21:47:14.457583+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457583+0000","last_undegraded":"2024-04-26T21:47:14.457583+0000","last_fullsized":"2024-04-26T21:47:14.457583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481460+0000","last_change":"2024-04-26T21:47:14.476244+0000","last_active":"2024-04-26T21:47:14.481460+0000","last_peered":"2024-04-26T21:47:14.481460+0000","last_clean":"2024-04-26T21:47:14.481460+0000","last_became_active":"2024-04-26T21:47:14.475929+0000","last_became_peered":"2024-04-26T21:47:14.475929+0000","last_unstale":"2024-04-26T21:47:14.481460+0000","last_undegraded":"2024-04-26T21:47:14.481460+0000","last_fullsized":"2024-04-26T21:47:14.481460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465106+0000","last_change":"2024-04-26T21:47:14.465106+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465106+0000","last_undegraded":"2024-04-26T21:47:14.465106+0000","last_fullsized":"2024-04-26T21:47:14.465106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486639+0000","last_change":"2024-04-26T21:47:14.479190+0000","last_active":"2024-04-26T21:47:14.486639+0000","last_peered":"2024-04-26T21:47:14.486639+0000","last_clean":"2024-04-26T21:47:14.486639+0000","last_became_active":"2024-04-26T21:47:14.478861+0000","last_became_peered":"2024-04-26T21:47:14.478861+0000","last_unstale":"2024-04-26T21:47:14.486639+0000","last_undegraded":"2024-04-26T21:47:14.486639+0000","last_fullsized":"2024-04-26T21:47:14.486639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468034+0000","last_change":"2024-04-26T21:47:14.468034+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468034+0000","last_undegraded":"2024-04-26T21:47:14.468034+0000","last_fullsized":"2024-04-26T21:47:14.468034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.474178+0000","last_change":"2024-04-26T21:47:14.465459+0000","last_active":"2024-04-26T21:47:14.474178+0000","last_peered":"2024-04-26T21:47:14.474178+0000","last_clean":"2024-04-26T21:47:14.474178+0000","last_became_active":"2024-04-26T21:47:14.464667+0000","last_became_peered":"2024-04-26T21:47:14.464667+0000","last_unstale":"2024-04-26T21:47:14.474178+0000","last_undegraded":"2024-04-26T21:47:14.474178+0000","last_fullsized":"2024-04-26T21:47:14.474178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470788+0000","last_change":"2024-04-26T21:47:14.470788+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470788+0000","last_undegraded":"2024-04-26T21:47:14.470788+0000","last_fullsized":"2024-04-26T21:47:14.470788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489741+0000","last_change":"2024-04-26T21:47:14.481643+0000","last_active":"2024-04-26T21:47:14.489741+0000","last_peered":"2024-04-26T21:47:14.489741+0000","last_clean":"2024-04-26T21:47:14.489741+0000","last_became_active":"2024-04-26T21:47:14.481119+0000","last_became_peered":"2024-04-26T21:47:14.481119+0000","last_unstale":"2024-04-26T21:47:14.489741+0000","last_undegraded":"2024-04-26T21:47:14.489741+0000","last_fullsized":"2024-04-26T21:47:14.489741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470626+0000","last_change":"2024-04-26T21:47:14.470626+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470626+0000","last_undegraded":"2024-04-26T21:47:14.470626+0000","last_fullsized":"2024-04-26T21:47:14.470626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481555+0000","last_change":"2024-04-26T21:47:14.475520+0000","last_active":"2024-04-26T21:47:14.481555+0000","last_peered":"2024-04-26T21:47:14.481555+0000","last_clean":"2024-04-26T21:47:14.481555+0000","last_became_active":"2024-04-26T21:47:14.475208+0000","last_became_peered":"2024-04-26T21:47:14.475208+0000","last_unstale":"2024-04-26T21:47:14.481555+0000","last_undegraded":"2024-04-26T21:47:14.481555+0000","last_fullsized":"2024-04-26T21:47:14.481555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467031+0000","last_change":"2024-04-26T21:47:14.467031+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467031+0000","last_undegraded":"2024-04-26T21:47:14.467031+0000","last_fullsized":"2024-04-26T21:47:14.467031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481067+0000","last_change":"2024-04-26T21:47:14.477761+0000","last_active":"2024-04-26T21:47:14.481067+0000","last_peered":"2024-04-26T21:47:14.481067+0000","last_clean":"2024-04-26T21:47:14.481067+0000","last_became_active":"2024-04-26T21:47:14.477292+0000","last_became_peered":"2024-04-26T21:47:14.477292+0000","last_unstale":"2024-04-26T21:47:14.481067+0000","last_undegraded":"2024-04-26T21:47:14.481067+0000","last_fullsized":"2024-04-26T21:47:14.481067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459408+0000","last_change":"2024-04-26T21:47:14.459408+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459408+0000","last_undegraded":"2024-04-26T21:47:14.459408+0000","last_fullsized":"2024-04-26T21:47:14.459408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490408+0000","last_change":"2024-04-26T21:47:14.485177+0000","last_active":"2024-04-26T21:47:14.490408+0000","last_peered":"2024-04-26T21:47:14.490408+0000","last_clean":"2024-04-26T21:47:14.490408+0000","last_became_active":"2024-04-26T21:47:14.483874+0000","last_became_peered":"2024-04-26T21:47:14.483874+0000","last_unstale":"2024-04-26T21:47:14.490408+0000","last_undegraded":"2024-04-26T21:47:14.490408+0000","last_fullsized":"2024-04-26T21:47:14.490408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450014+0000","last_change":"2024-04-26T21:47:14.450014+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450014+0000","last_undegraded":"2024-04-26T21:47:14.450014+0000","last_fullsized":"2024-04-26T21:47:14.450014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485168+0000","last_change":"2024-04-26T21:47:14.480541+0000","last_active":"2024-04-26T21:47:14.485168+0000","last_peered":"2024-04-26T21:47:14.485168+0000","last_clean":"2024-04-26T21:47:14.485168+0000","last_became_active":"2024-04-26T21:47:14.480243+0000","last_became_peered":"2024-04-26T21:47:14.480243+0000","last_unstale":"2024-04-26T21:47:14.485168+0000","last_undegraded":"2024-04-26T21:47:14.485168+0000","last_fullsized":"2024-04-26T21:47:14.485168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:09.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.459791+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:14.459791+0000","last_peered":"2024-04-26T21:47:14.459791+0000","last_clean":"2024-04-26T21:47:14.459791+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:14.459791+0000","last_undegraded":"2024-04-26T21:47:14.459791+0000","last_fullsized":"2024-04-26T21:47:14.459791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:19.932 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:20.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.172+0000 7ff76fe00640 1 Processor -- start 2024-04-26T21:47:20.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.172+0000 7ff76fe00640 1 -- start start 2024-04-26T21:47:20.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.173+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff7700d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.173+0000 7ff76fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7700b3b20 0x7ff7700b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.173+0000 7ff76fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff770146a60 con 0x7ff770146680 2024-04-26T21:47:20.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.173+0000 7ff76fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff77006e970 con 0x7ff7700b3b20 2024-04-26T21:47:20.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.173+0000 7ff76fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff7700b60a0 con 0x7ff7700d0e70 2024-04-26T21:47:20.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.174+0000 7ff767e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff7700d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.174+0000 7ff76f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7700b3b20 0x7ff7700b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.174+0000 7ff76ea00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff770146680 0x7ff7700b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:52176/0 (socket says 172.21.15.106:52176) 2024-04-26T21:47:20.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.174+0000 7ff76ea00640 1 -- 172.21.15.106:0/3730482430 learned_addr learned my addr 172.21.15.106:0/3730482430 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff76f400640 1 -- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 msgr2=0x7ff7700d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff76f400640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff7700d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff76f400640 1 -- 172.21.15.106:0/3730482430 >> v1:172.21.15.204:6789/0 conn(0x7ff770146680 legacy=0x7ff7700b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff76f400640 1 -- 172.21.15.106:0/3730482430 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77006dd60 con 0x7ff7700b3b20 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff76e000640 1 -- 172.21.15.106:0/3730482430 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084150957 0 0) 0x7ff770146a60 con 0x7ff770146680 2024-04-26T21:47:20.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.175+0000 7ff767e00640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff7700d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:20.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.177+0000 7ff76f400640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7700b3fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff76802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871372a07f4999fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.177+0000 7ff76e000640 1 -- 172.21.15.106:0/3730482430 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768007740 con 0x7ff7700b3b20 2024-04-26T21:47:20.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.177+0000 7ff76e000640 1 -- 172.21.15.106:0/3730482430 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7680078e0 con 0x7ff7700b3b20 2024-04-26T21:47:20.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.177+0000 7ff76e000640 1 -- 172.21.15.106:0/3730482430 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768007cb0 con 0x7ff7700b3b20 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 -- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 msgr2=0x7ff7700b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7700b3fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff76802a660 comp rx=0 tx=0).stop 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 -- 172.21.15.106:0/3730482430 shutdown_connections 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7700b3fb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 --2- 172.21.15.106:0/3730482430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff7700d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 -- 172.21.15.106:0/3730482430 >> 172.21.15.106:0/3730482430 conn(0x7ff77006c740 msgr2=0x7ff77006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 -- 172.21.15.106:0/3730482430 shutdown_connections 2024-04-26T21:47:20.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.178+0000 7ff76fe00640 1 -- 172.21.15.106:0/3730482430 wait complete. 2024-04-26T21:47:20.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.180+0000 7ff76fe00640 1 Processor -- start 2024-04-26T21:47:20.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.180+0000 7ff76fe00640 1 -- start start 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7701396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff770139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff77006e320 con 0x7ff770146680 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700d17f0 con 0x7ff7700b3b20 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff77006e600 con 0x7ff7700d0e70 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53520/0 (socket says 172.21.15.106:53520) 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76f400640 1 -- 172.21.15.106:0/2682491677 learned_addr learned my addr 172.21.15.106:0/2682491677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.181+0000 7ff76ea00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7701396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff767e00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff770139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 -- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 msgr2=0x7ff770139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff770139df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 -- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 msgr2=0x7ff7701396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7701396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 -- 172.21.15.106:0/2682491677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7700736a0 con 0x7ff770146680 2024-04-26T21:47:20.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff767e00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff770139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:20.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76f400640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768040070 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff768031c40 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768009450 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.182+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77013e0e0 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.183+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77013e6a0 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.184+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7700693c0 con 0x7ff770146680 2024-04-26T21:47:20.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.184+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff768039040 con 0x7ff770146680 2024-04-26T21:47:20.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.185+0000 7ff766a00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.186+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff7680acb00 con 0x7ff770146680 2024-04-26T21:47:20.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.186+0000 7ff76ea00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.186+0000 7ff76ea00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 0x7ff74c06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff760000f60 tx=0x7ff760007d00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.190+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff76807ae00 con 0x7ff770146680 2024-04-26T21:47:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.381+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff770145d00 con 0x7ff74c06c8f0 2024-04-26T21:47:20.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.400+0000 7ff766a00640 1 -- 172.21.15.106:0/2682491677 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7ff770145d00 con 0x7ff74c06c8f0 2024-04-26T21:47:20.404 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:20.405 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 msgr2=0x7ff74c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 0x7ff74c06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff760000f60 tx=0x7ff760007d00 comp rx=0 tx=0).stop 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 msgr2=0x7ff77013a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff768004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 shutdown_connections 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff770146680 0x7ff77013a330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7700d0e70 0x7ff770139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 --2- 172.21.15.106:0/2682491677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7700b3b20 0x7ff7701396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.406+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 >> 172.21.15.106:0/2682491677 conn(0x7ff77006c740 msgr2=0x7ff7700d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.407+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 shutdown_connections 2024-04-26T21:47:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.407+0000 7ff76fe00640 1 -- 172.21.15.106:0/2682491677 wait complete. 2024-04-26T21:47:20.425 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T21:47:19.113548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220020,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754700,"statfs":{"total":773094113280,"available":772868812800,"internally_reserved":0,"allocated":3936256,"data_stored":2230199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.676889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469233+0000","last_change":"2024-04-26T21:47:14.469233+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469233+0000","last_undegraded":"2024-04-26T21:47:14.469233+0000","last_fullsized":"2024-04-26T21:47:14.469233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.473349+0000","last_change":"2024-04-26T21:47:14.473349+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.473349+0000","last_undegraded":"2024-04-26T21:47:14.473349+0000","last_fullsized":"2024-04-26T21:47:14.473349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466695+0000","last_change":"2024-04-26T21:47:14.466695+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466695+0000","last_undegraded":"2024-04-26T21:47:14.466695+0000","last_fullsized":"2024-04-26T21:47:14.466695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463852+0000","last_change":"2024-04-26T21:47:14.463852+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463852+0000","last_undegraded":"2024-04-26T21:47:14.463852+0000","last_fullsized":"2024-04-26T21:47:14.463852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466007+0000","last_change":"2024-04-26T21:47:14.466007+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466007+0000","last_undegraded":"2024-04-26T21:47:14.466007+0000","last_fullsized":"2024-04-26T21:47:14.466007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.471474+0000","last_change":"2024-04-26T21:47:14.471474+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.471474+0000","last_undegraded":"2024-04-26T21:47:14.471474+0000","last_fullsized":"2024-04-26T21:47:14.471474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452768+0000","last_change":"2024-04-26T21:47:14.452768+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452768+0000","last_undegraded":"2024-04-26T21:47:14.452768+0000","last_fullsized":"2024-04-26T21:47:14.452768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457237+0000","last_change":"2024-04-26T21:47:14.457237+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457237+0000","last_undegraded":"2024-04-26T21:47:14.457237+0000","last_fullsized":"2024-04-26T21:47:14.457237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453766+0000","last_change":"2024-04-26T21:47:14.453766+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453766+0000","last_undegraded":"2024-04-26T21:47:14.453766+0000","last_fullsized":"2024-04-26T21:47:14.453766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462059+0000","last_change":"2024-04-26T21:47:14.462059+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462059+0000","last_undegraded":"2024-04-26T21:47:14.462059+0000","last_fullsized":"2024-04-26T21:47:14.462059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453846+0000","last_change":"2024-04-26T21:47:14.453846+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453846+0000","last_undegraded":"2024-04-26T21:47:14.453846+0000","last_fullsized":"2024-04-26T21:47:14.453846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465401+0000","last_change":"2024-04-26T21:47:14.465401+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465401+0000","last_undegraded":"2024-04-26T21:47:14.465401+0000","last_fullsized":"2024-04-26T21:47:14.465401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459374+0000","last_change":"2024-04-26T21:47:14.459374+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459374+0000","last_undegraded":"2024-04-26T21:47:14.459374+0000","last_fullsized":"2024-04-26T21:47:14.459374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470943+0000","last_change":"2024-04-26T21:47:14.470943+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470943+0000","last_undegraded":"2024-04-26T21:47:14.470943+0000","last_fullsized":"2024-04-26T21:47:14.470943+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470934+0000","last_change":"2024-04-26T21:47:14.470934+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470934+0000","last_undegraded":"2024-04-26T21:47:14.470934+0000","last_fullsized":"2024-04-26T21:47:14.470934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449141+0000","last_change":"2024-04-26T21:47:14.449141+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449141+0000","last_undegraded":"2024-04-26T21:47:14.449141+0000","last_fullsized":"2024-04-26T21:47:14.449141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450358+0000","last_change":"2024-04-26T21:47:14.450358+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450358+0000","last_undegraded":"2024-04-26T21:47:14.450358+0000","last_fullsized":"2024-04-26T21:47:14.450358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470776+0000","last_change":"2024-04-26T21:47:14.470776+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470776+0000","last_undegraded":"2024-04-26T21:47:14.470776+0000","last_fullsized":"2024-04-26T21:47:14.470776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469285+0000","last_change":"2024-04-26T21:47:14.469285+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469285+0000","last_undegraded":"2024-04-26T21:47:14.469285+0000","last_fullsized":"2024-04-26T21:47:14.469285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457654+0000","last_change":"2024-04-26T21:47:14.457654+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457654+0000","last_undegraded":"2024-04-26T21:47:14.457654+0000","last_fullsized":"2024-04-26T21:47:14.457654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457377+0000","last_change":"2024-04-26T21:47:14.457377+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457377+0000","last_undegraded":"2024-04-26T21:47:14.457377+0000","last_fullsized":"2024-04-26T21:47:14.457377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452220+0000","last_change":"2024-04-26T21:47:14.452220+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452220+0000","last_undegraded":"2024-04-26T21:47:14.452220+0000","last_fullsized":"2024-04-26T21:47:14.452220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462364+0000","last_change":"2024-04-26T21:47:14.462364+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462364+0000","last_undegraded":"2024-04-26T21:47:14.462364+0000","last_fullsized":"2024-04-26T21:47:14.462364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469923+0000","last_change":"2024-04-26T21:47:14.469923+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469923+0000","last_undegraded":"2024-04-26T21:47:14.469923+0000","last_fullsized":"2024-04-26T21:47:14.469923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.460700+0000","last_change":"2024-04-26T21:47:14.460700+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.460700+0000","last_undegraded":"2024-04-26T21:47:14.460700+0000","last_fullsized":"2024-04-26T21:47:14.460700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468882+0000","last_change":"2024-04-26T21:47:14.468882+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468882+0000","last_undegraded":"2024-04-26T21:47:14.468882+0000","last_fullsized":"2024-04-26T21:47:14.468882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465607+0000","last_change":"2024-04-26T21:47:14.465607+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465607+0000","last_undegraded":"2024-04-26T21:47:14.465607+0000","last_fullsized":"2024-04-26T21:47:14.465607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463389+0000","last_change":"2024-04-26T21:47:14.463389+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463389+0000","last_undegraded":"2024-04-26T21:47:14.463389+0000","last_fullsized":"2024-04-26T21:47:14.463389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456256+0000","last_change":"2024-04-26T21:47:14.456256+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456256+0000","last_undegraded":"2024-04-26T21:47:14.456256+0000","last_fullsized":"2024-04-26T21:47:14.456256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450566+0000","last_change":"2024-04-26T21:47:14.450566+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450566+0000","last_undegraded":"2024-04-26T21:47:14.450566+0000","last_fullsized":"2024-04-26T21:47:14.450566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468593+0000","last_change":"2024-04-26T21:47:14.468593+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468593+0000","last_undegraded":"2024-04-26T21:47:14.468593+0000","last_fullsized":"2024-04-26T21:47:14.468593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456198+0000","last_change":"2024-04-26T21:47:14.456198+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456198+0000","last_undegraded":"2024-04-26T21:47:14.456198+0000","last_fullsized":"2024-04-26T21:47:14.456198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470298+0000","last_change":"2024-04-26T21:47:14.470298+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470298+0000","last_undegraded":"2024-04-26T21:47:14.470298+0000","last_fullsized":"2024-04-26T21:47:14.470298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488061+0000","last_change":"2024-04-26T21:47:14.483471+0000","last_active":"2024-04-26T21:47:14.488061+0000","last_peered":"2024-04-26T21:47:14.488061+0000","last_clean":"2024-04-26T21:47:14.488061+0000","last_became_active":"2024-04-26T21:47:14.483132+0000","last_became_peered":"2024-04-26T21:47:14.483132+0000","last_unstale":"2024-04-26T21:47:14.488061+0000","last_undegraded":"2024-04-26T21:47:14.488061+0000","last_fullsized":"2024-04-26T21:47:14.488061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468665+0000","last_change":"2024-04-26T21:47:14.468665+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468665+0000","last_undegraded":"2024-04-26T21:47:14.468665+0000","last_fullsized":"2024-04-26T21:47:14.468665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486611+0000","last_change":"2024-04-26T21:47:14.477793+0000","last_active":"2024-04-26T21:47:14.486611+0000","last_peered":"2024-04-26T21:47:14.486611+0000","last_clean":"2024-04-26T21:47:14.486611+0000","last_became_active":"2024-04-26T21:47:14.477460+0000","last_became_peered":"2024-04-26T21:47:14.477460+0000","last_unstale":"2024-04-26T21:47:14.486611+0000","last_undegraded":"2024-04-26T21:47:14.486611+0000","last_fullsized":"2024-04-26T21:47:14.486611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453644+0000","last_change":"2024-04-26T21:47:14.453644+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453644+0000","last_undegraded":"2024-04-26T21:47:14.453644+0000","last_fullsized":"2024-04-26T21:47:14.453644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490372+0000","last_change":"2024-04-26T21:47:14.484669+0000","last_active":"2024-04-26T21:47:14.490372+0000","last_peered":"2024-04-26T21:47:14.490372+0000","last_clean":"2024-04-26T21:47:14.490372+0000","last_became_active":"2024-04-26T21:47:14.483933+0000","last_became_peered":"2024-04-26T21:47:14.483933+0000","last_unstale":"2024-04-26T21:47:14.490372+0000","last_undegraded":"2024-04-26T21:47:14.490372+0000","last_fullsized":"2024-04-26T21:47:14.490372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463630+0000","last_change":"2024-04-26T21:47:14.463630+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463630+0000","last_undegraded":"2024-04-26T21:47:14.463630+0000","last_fullsized":"2024-04-26T21:47:14.463630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.469248+0000","last_change":"2024-04-26T21:47:14.462571+0000","last_active":"2024-04-26T21:47:14.469248+0000","last_peered":"2024-04-26T21:47:14.469248+0000","last_clean":"2024-04-26T21:47:14.469248+0000","last_became_active":"2024-04-26T21:47:14.461470+0000","last_became_peered":"2024-04-26T21:47:14.461470+0000","last_unstale":"2024-04-26T21:47:14.469248+0000","last_undegraded":"2024-04-26T21:47:14.469248+0000","last_fullsized":"2024-04-26T21:47:14.469248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449275+0000","last_change":"2024-04-26T21:47:14.449275+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449275+0000","last_undegraded":"2024-04-26T21:47:14.449275+0000","last_fullsized":"2024-04-26T21:47:14.449275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490646+0000","last_change":"2024-04-26T21:47:14.485718+0000","last_active":"2024-04-26T21:47:14.490646+0000","last_peered":"2024-04-26T21:47:14.490646+0000","last_clean":"2024-04-26T21:47:14.490646+0000","last_became_active":"2024-04-26T21:47:14.485458+0000","last_became_peered":"2024-04-26T21:47:14.485458+0000","last_unstale":"2024-04-26T21:47:14.490646+0000","last_undegraded":"2024-04-26T21:47:14.490646+0000","last_fullsized":"2024-04-26T21:47:14.490646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:12.839326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463772+0000","last_change":"2024-04-26T21:47:14.463772+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463772+0000","last_undegraded":"2024-04-26T21:47:14.463772+0000","last_fullsized":"2024-04-26T21:47:14.463772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488020+0000","last_change":"2024-04-26T21:47:14.484537+0000","last_active":"2024-04-26T21:47:14.488020+0000","last_peered":"2024-04-26T21:47:14.488020+0000","last_clean":"2024-04-26T21:47:14.488020+0000","last_became_active":"2024-04-26T21:47:14.484218+0000","last_became_peered":"2024-04-26T21:47:14.484218+0000","last_unstale":"2024-04-26T21:47:14.488020+0000","last_undegraded":"2024-04-26T21:47:14.488020+0000","last_fullsized":"2024-04-26T21:47:14.488020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:23.713847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452619+0000","last_change":"2024-04-26T21:47:14.452619+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452619+0000","last_undegraded":"2024-04-26T21:47:14.452619+0000","last_fullsized":"2024-04-26T21:47:14.452619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490039+0000","last_change":"2024-04-26T21:47:14.483496+0000","last_active":"2024-04-26T21:47:14.490039+0000","last_peered":"2024-04-26T21:47:14.490039+0000","last_clean":"2024-04-26T21:47:14.490039+0000","last_became_active":"2024-04-26T21:47:14.483201+0000","last_became_peered":"2024-04-26T21:47:14.483201+0000","last_unstale":"2024-04-26T21:47:14.490039+0000","last_undegraded":"2024-04-26T21:47:14.490039+0000","last_fullsized":"2024-04-26T21:47:14.490039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463534+0000","last_change":"2024-04-26T21:47:14.463534+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463534+0000","last_undegraded":"2024-04-26T21:47:14.463534+0000","last_fullsized":"2024-04-26T21:47:14.463534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482982+0000","last_change":"2024-04-26T21:47:14.480135+0000","last_active":"2024-04-26T21:47:14.482982+0000","last_peered":"2024-04-26T21:47:14.482982+0000","last_clean":"2024-04-26T21:47:14.482982+0000","last_became_active":"2024-04-26T21:47:14.479762+0000","last_became_peered":"2024-04-26T21:47:14.479762+0000","last_unstale":"2024-04-26T21:47:14.482982+0000","last_undegraded":"2024-04-26T21:47:14.482982+0000","last_fullsized":"2024-04-26T21:47:14.482982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:37.394548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463597+0000","last_change":"2024-04-26T21:47:14.463597+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463597+0000","last_undegraded":"2024-04-26T21:47:14.463597+0000","last_fullsized":"2024-04-26T21:47:14.463597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490144+0000","last_change":"2024-04-26T21:47:14.484137+0000","last_active":"2024-04-26T21:47:14.490144+0000","last_peered":"2024-04-26T21:47:14.490144+0000","last_clean":"2024-04-26T21:47:14.490144+0000","last_became_active":"2024-04-26T21:47:14.483547+0000","last_became_peered":"2024-04-26T21:47:14.483547+0000","last_unstale":"2024-04-26T21:47:14.490144+0000","last_undegraded":"2024-04-26T21:47:14.490144+0000","last_fullsized":"2024-04-26T21:47:14.490144+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.476712+0000","last_change":"2024-04-26T21:47:14.476712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.476712+0000","last_undegraded":"2024-04-26T21:47:14.476712+0000","last_fullsized":"2024-04-26T21:47:14.476712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489763+0000","last_change":"2024-04-26T21:47:14.485542+0000","last_active":"2024-04-26T21:47:14.489763+0000","last_peered":"2024-04-26T21:47:14.489763+0000","last_clean":"2024-04-26T21:47:14.489763+0000","last_became_active":"2024-04-26T21:47:14.485231+0000","last_became_peered":"2024-04-26T21:47:14.485231+0000","last_unstale":"2024-04-26T21:47:14.489763+0000","last_undegraded":"2024-04-26T21:47:14.489763+0000","last_fullsized":"2024-04-26T21:47:14.489763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.475921+0000","last_change":"2024-04-26T21:47:14.475921+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.475921+0000","last_undegraded":"2024-04-26T21:47:14.475921+0000","last_fullsized":"2024-04-26T21:47:14.475921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.480096+0000","last_change":"2024-04-26T21:47:14.472755+0000","last_active":"2024-04-26T21:47:14.480096+0000","last_peered":"2024-04-26T21:47:14.480096+0000","last_clean":"2024-04-26T21:47:14.480096+0000","last_became_active":"2024-04-26T21:47:14.472547+0000","last_became_peered":"2024-04-26T21:47:14.472547+0000","last_unstale":"2024-04-26T21:47:14.480096+0000","last_undegraded":"2024-04-26T21:47:14.480096+0000","last_fullsized":"2024-04-26T21:47:14.480096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467306+0000","last_change":"2024-04-26T21:47:14.467306+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467306+0000","last_undegraded":"2024-04-26T21:47:14.467306+0000","last_fullsized":"2024-04-26T21:47:14.467306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493863+0000","last_change":"2024-04-26T21:47:14.487095+0000","last_active":"2024-04-26T21:47:14.493863+0000","last_peered":"2024-04-26T21:47:14.493863+0000","last_clean":"2024-04-26T21:47:14.493863+0000","last_became_active":"2024-04-26T21:47:14.486802+0000","last_became_peered":"2024-04-26T21:47:14.486802+0000","last_unstale":"2024-04-26T21:47:14.493863+0000","last_undegraded":"2024-04-26T21:47:14.493863+0000","last_fullsized":"2024-04-26T21:47:14.493863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.461282+0000","last_change":"2024-04-26T21:47:14.461282+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.461282+0000","last_undegraded":"2024-04-26T21:47:14.461282+0000","last_fullsized":"2024-04-26T21:47:14.461282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481227+0000","last_change":"2024-04-26T21:47:14.478529+0000","last_active":"2024-04-26T21:47:14.481227+0000","last_peered":"2024-04-26T21:47:14.481227+0000","last_clean":"2024-04-26T21:47:14.481227+0000","last_became_active":"2024-04-26T21:47:14.478281+0000","last_became_peered":"2024-04-26T21:47:14.478281+0000","last_unstale":"2024-04-26T21:47:14.481227+0000","last_undegraded":"2024-04-26T21:47:14.481227+0000","last_fullsized":"2024-04-26T21:47:14.481227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:02.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465187+0000","last_change":"2024-04-26T21:47:14.465187+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465187+0000","last_undegraded":"2024-04-26T21:47:14.465187+0000","last_fullsized":"2024-04-26T21:47:14.465187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481931+0000","last_change":"2024-04-26T21:47:14.480124+0000","last_active":"2024-04-26T21:47:14.481931+0000","last_peered":"2024-04-26T21:47:14.481931+0000","last_clean":"2024-04-26T21:47:14.481931+0000","last_became_active":"2024-04-26T21:47:14.479846+0000","last_became_peered":"2024-04-26T21:47:14.479846+0000","last_unstale":"2024-04-26T21:47:14.481931+0000","last_undegraded":"2024-04-26T21:47:14.481931+0000","last_fullsized":"2024-04-26T21:47:14.481931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.455199+0000","last_change":"2024-04-26T21:47:14.455199+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.455199+0000","last_undegraded":"2024-04-26T21:47:14.455199+0000","last_fullsized":"2024-04-26T21:47:14.455199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490380+0000","last_change":"2024-04-26T21:47:14.485004+0000","last_active":"2024-04-26T21:47:14.490380+0000","last_peered":"2024-04-26T21:47:14.490380+0000","last_clean":"2024-04-26T21:47:14.490380+0000","last_became_active":"2024-04-26T21:47:14.483355+0000","last_became_peered":"2024-04-26T21:47:14.483355+0000","last_unstale":"2024-04-26T21:47:14.490380+0000","last_undegraded":"2024-04-26T21:47:14.490380+0000","last_fullsized":"2024-04-26T21:47:14.490380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467945+0000","last_change":"2024-04-26T21:47:14.467945+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467945+0000","last_undegraded":"2024-04-26T21:47:14.467945+0000","last_fullsized":"2024-04-26T21:47:14.467945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487389+0000","last_change":"2024-04-26T21:47:14.482285+0000","last_active":"2024-04-26T21:47:14.487389+0000","last_peered":"2024-04-26T21:47:14.487389+0000","last_clean":"2024-04-26T21:47:14.487389+0000","last_became_active":"2024-04-26T21:47:14.481952+0000","last_became_peered":"2024-04-26T21:47:14.481952+0000","last_unstale":"2024-04-26T21:47:14.487389+0000","last_undegraded":"2024-04-26T21:47:14.487389+0000","last_fullsized":"2024-04-26T21:47:14.487389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456504+0000","last_change":"2024-04-26T21:47:14.456504+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456504+0000","last_undegraded":"2024-04-26T21:47:14.456504+0000","last_fullsized":"2024-04-26T21:47:14.456504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493983+0000","last_change":"2024-04-26T21:47:14.485819+0000","last_active":"2024-04-26T21:47:14.493983+0000","last_peered":"2024-04-26T21:47:14.493983+0000","last_clean":"2024-04-26T21:47:14.493983+0000","last_became_active":"2024-04-26T21:47:14.485435+0000","last_became_peered":"2024-04-26T21:47:14.485435+0000","last_unstale":"2024-04-26T21:47:14.493983+0000","last_undegraded":"2024-04-26T21:47:14.493983+0000","last_fullsized":"2024-04-26T21:47:14.493983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465748+0000","last_change":"2024-04-26T21:47:14.465748+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465748+0000","last_undegraded":"2024-04-26T21:47:14.465748+0000","last_fullsized":"2024-04-26T21:47:14.465748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485819+0000","last_change":"2024-04-26T21:47:14.481662+0000","last_active":"2024-04-26T21:47:14.485819+0000","last_peered":"2024-04-26T21:47:14.485819+0000","last_clean":"2024-04-26T21:47:14.485819+0000","last_became_active":"2024-04-26T21:47:14.480582+0000","last_became_peered":"2024-04-26T21:47:14.480582+0000","last_unstale":"2024-04-26T21:47:14.485819+0000","last_undegraded":"2024-04-26T21:47:14.485819+0000","last_fullsized":"2024-04-26T21:47:14.485819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:00.441765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462995+0000","last_change":"2024-04-26T21:47:14.462995+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462995+0000","last_undegraded":"2024-04-26T21:47:14.462995+0000","last_fullsized":"2024-04-26T21:47:14.462995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478547+0000","last_change":"2024-04-26T21:47:14.475423+0000","last_active":"2024-04-26T21:47:14.478547+0000","last_peered":"2024-04-26T21:47:14.478547+0000","last_clean":"2024-04-26T21:47:14.478547+0000","last_became_active":"2024-04-26T21:47:14.475083+0000","last_became_peered":"2024-04-26T21:47:14.475083+0000","last_unstale":"2024-04-26T21:47:14.478547+0000","last_undegraded":"2024-04-26T21:47:14.478547+0000","last_fullsized":"2024-04-26T21:47:14.478547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459712+0000","last_change":"2024-04-26T21:47:14.459712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459712+0000","last_undegraded":"2024-04-26T21:47:14.459712+0000","last_fullsized":"2024-04-26T21:47:14.459712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490682+0000","last_change":"2024-04-26T21:47:14.487648+0000","last_active":"2024-04-26T21:47:14.490682+0000","last_peered":"2024-04-26T21:47:14.490682+0000","last_clean":"2024-04-26T21:47:14.490682+0000","last_became_active":"2024-04-26T21:47:14.487356+0000","last_became_peered":"2024-04-26T21:47:14.487356+0000","last_unstale":"2024-04-26T21:47:14.490682+0000","last_undegraded":"2024-04-26T21:47:14.490682+0000","last_fullsized":"2024-04-26T21:47:14.490682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:02.916803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458838+0000","last_change":"2024-04-26T21:47:14.458838+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458838+0000","last_undegraded":"2024-04-26T21:47:14.458838+0000","last_fullsized":"2024-04-26T21:47:14.458838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486377+0000","last_change":"2024-04-26T21:47:14.478983+0000","last_active":"2024-04-26T21:47:14.486377+0000","last_peered":"2024-04-26T21:47:14.486377+0000","last_clean":"2024-04-26T21:47:14.486377+0000","last_became_active":"2024-04-26T21:47:14.478698+0000","last_became_peered":"2024-04-26T21:47:14.478698+0000","last_unstale":"2024-04-26T21:47:14.486377+0000","last_undegraded":"2024-04-26T21:47:14.486377+0000","last_fullsized":"2024-04-26T21:47:14.486377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458086+0000","last_change":"2024-04-26T21:47:14.458086+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458086+0000","last_undegraded":"2024-04-26T21:47:14.458086+0000","last_fullsized":"2024-04-26T21:47:14.458086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487855+0000","last_change":"2024-04-26T21:47:14.482597+0000","last_active":"2024-04-26T21:47:14.487855+0000","last_peered":"2024-04-26T21:47:14.487855+0000","last_clean":"2024-04-26T21:47:14.487855+0000","last_became_active":"2024-04-26T21:47:14.482299+0000","last_became_peered":"2024-04-26T21:47:14.482299+0000","last_unstale":"2024-04-26T21:47:14.487855+0000","last_undegraded":"2024-04-26T21:47:14.487855+0000","last_fullsized":"2024-04-26T21:47:14.487855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:20.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457559+0000","last_change":"2024-04-26T21:47:14.457559+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457559+0000","last_undegraded":"2024-04-26T21:47:14.457559+0000","last_fullsized":"2024-04-26T21:47:14.457559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478807+0000","last_change":"2024-04-26T21:47:14.475056+0000","last_active":"2024-04-26T21:47:14.478807+0000","last_peered":"2024-04-26T21:47:14.478807+0000","last_clean":"2024-04-26T21:47:14.478807+0000","last_became_active":"2024-04-26T21:47:14.474735+0000","last_became_peered":"2024-04-26T21:47:14.474735+0000","last_unstale":"2024-04-26T21:47:14.478807+0000","last_undegraded":"2024-04-26T21:47:14.478807+0000","last_fullsized":"2024-04-26T21:47:14.478807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:06.758168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453724+0000","last_change":"2024-04-26T21:47:14.453724+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453724+0000","last_undegraded":"2024-04-26T21:47:14.453724+0000","last_fullsized":"2024-04-26T21:47:14.453724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482671+0000","last_change":"2024-04-26T21:47:14.478761+0000","last_active":"2024-04-26T21:47:14.482671+0000","last_peered":"2024-04-26T21:47:14.482671+0000","last_clean":"2024-04-26T21:47:14.482671+0000","last_became_active":"2024-04-26T21:47:14.477387+0000","last_became_peered":"2024-04-26T21:47:14.477387+0000","last_unstale":"2024-04-26T21:47:14.482671+0000","last_undegraded":"2024-04-26T21:47:14.482671+0000","last_fullsized":"2024-04-26T21:47:14.482671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457583+0000","last_change":"2024-04-26T21:47:14.457583+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457583+0000","last_undegraded":"2024-04-26T21:47:14.457583+0000","last_fullsized":"2024-04-26T21:47:14.457583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481460+0000","last_change":"2024-04-26T21:47:14.476244+0000","last_active":"2024-04-26T21:47:14.481460+0000","last_peered":"2024-04-26T21:47:14.481460+0000","last_clean":"2024-04-26T21:47:14.481460+0000","last_became_active":"2024-04-26T21:47:14.475929+0000","last_became_peered":"2024-04-26T21:47:14.475929+0000","last_unstale":"2024-04-26T21:47:14.481460+0000","last_undegraded":"2024-04-26T21:47:14.481460+0000","last_fullsized":"2024-04-26T21:47:14.481460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465106+0000","last_change":"2024-04-26T21:47:14.465106+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465106+0000","last_undegraded":"2024-04-26T21:47:14.465106+0000","last_fullsized":"2024-04-26T21:47:14.465106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486639+0000","last_change":"2024-04-26T21:47:14.479190+0000","last_active":"2024-04-26T21:47:14.486639+0000","last_peered":"2024-04-26T21:47:14.486639+0000","last_clean":"2024-04-26T21:47:14.486639+0000","last_became_active":"2024-04-26T21:47:14.478861+0000","last_became_peered":"2024-04-26T21:47:14.478861+0000","last_unstale":"2024-04-26T21:47:14.486639+0000","last_undegraded":"2024-04-26T21:47:14.486639+0000","last_fullsized":"2024-04-26T21:47:14.486639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468034+0000","last_change":"2024-04-26T21:47:14.468034+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468034+0000","last_undegraded":"2024-04-26T21:47:14.468034+0000","last_fullsized":"2024-04-26T21:47:14.468034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.474178+0000","last_change":"2024-04-26T21:47:14.465459+0000","last_active":"2024-04-26T21:47:14.474178+0000","last_peered":"2024-04-26T21:47:14.474178+0000","last_clean":"2024-04-26T21:47:14.474178+0000","last_became_active":"2024-04-26T21:47:14.464667+0000","last_became_peered":"2024-04-26T21:47:14.464667+0000","last_unstale":"2024-04-26T21:47:14.474178+0000","last_undegraded":"2024-04-26T21:47:14.474178+0000","last_fullsized":"2024-04-26T21:47:14.474178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470788+0000","last_change":"2024-04-26T21:47:14.470788+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470788+0000","last_undegraded":"2024-04-26T21:47:14.470788+0000","last_fullsized":"2024-04-26T21:47:14.470788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489741+0000","last_change":"2024-04-26T21:47:14.481643+0000","last_active":"2024-04-26T21:47:14.489741+0000","last_peered":"2024-04-26T21:47:14.489741+0000","last_clean":"2024-04-26T21:47:14.489741+0000","last_became_active":"2024-04-26T21:47:14.481119+0000","last_became_peered":"2024-04-26T21:47:14.481119+0000","last_unstale":"2024-04-26T21:47:14.489741+0000","last_undegraded":"2024-04-26T21:47:14.489741+0000","last_fullsized":"2024-04-26T21:47:14.489741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470626+0000","last_change":"2024-04-26T21:47:14.470626+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470626+0000","last_undegraded":"2024-04-26T21:47:14.470626+0000","last_fullsized":"2024-04-26T21:47:14.470626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481555+0000","last_change":"2024-04-26T21:47:14.475520+0000","last_active":"2024-04-26T21:47:14.481555+0000","last_peered":"2024-04-26T21:47:14.481555+0000","last_clean":"2024-04-26T21:47:14.481555+0000","last_became_active":"2024-04-26T21:47:14.475208+0000","last_became_peered":"2024-04-26T21:47:14.475208+0000","last_unstale":"2024-04-26T21:47:14.481555+0000","last_undegraded":"2024-04-26T21:47:14.481555+0000","last_fullsized":"2024-04-26T21:47:14.481555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467031+0000","last_change":"2024-04-26T21:47:14.467031+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467031+0000","last_undegraded":"2024-04-26T21:47:14.467031+0000","last_fullsized":"2024-04-26T21:47:14.467031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481067+0000","last_change":"2024-04-26T21:47:14.477761+0000","last_active":"2024-04-26T21:47:14.481067+0000","last_peered":"2024-04-26T21:47:14.481067+0000","last_clean":"2024-04-26T21:47:14.481067+0000","last_became_active":"2024-04-26T21:47:14.477292+0000","last_became_peered":"2024-04-26T21:47:14.477292+0000","last_unstale":"2024-04-26T21:47:14.481067+0000","last_undegraded":"2024-04-26T21:47:14.481067+0000","last_fullsized":"2024-04-26T21:47:14.481067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459408+0000","last_change":"2024-04-26T21:47:14.459408+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459408+0000","last_undegraded":"2024-04-26T21:47:14.459408+0000","last_fullsized":"2024-04-26T21:47:14.459408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490408+0000","last_change":"2024-04-26T21:47:14.485177+0000","last_active":"2024-04-26T21:47:14.490408+0000","last_peered":"2024-04-26T21:47:14.490408+0000","last_clean":"2024-04-26T21:47:14.490408+0000","last_became_active":"2024-04-26T21:47:14.483874+0000","last_became_peered":"2024-04-26T21:47:14.483874+0000","last_unstale":"2024-04-26T21:47:14.490408+0000","last_undegraded":"2024-04-26T21:47:14.490408+0000","last_fullsized":"2024-04-26T21:47:14.490408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450014+0000","last_change":"2024-04-26T21:47:14.450014+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450014+0000","last_undegraded":"2024-04-26T21:47:14.450014+0000","last_fullsized":"2024-04-26T21:47:14.450014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485168+0000","last_change":"2024-04-26T21:47:14.480541+0000","last_active":"2024-04-26T21:47:14.485168+0000","last_peered":"2024-04-26T21:47:14.485168+0000","last_clean":"2024-04-26T21:47:14.485168+0000","last_became_active":"2024-04-26T21:47:14.480243+0000","last_became_peered":"2024-04-26T21:47:14.480243+0000","last_unstale":"2024-04-26T21:47:14.485168+0000","last_undegraded":"2024-04-26T21:47:14.485168+0000","last_fullsized":"2024-04-26T21:47:14.485168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:09.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.459791+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:14.459791+0000","last_peered":"2024-04-26T21:47:14.459791+0000","last_clean":"2024-04-26T21:47:14.459791+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:14.459791+0000","last_undegraded":"2024-04-26T21:47:14.459791+0000","last_fullsized":"2024-04-26T21:47:14.459791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:20.429 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:20.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.676+0000 7fdfaa800640 1 Processor -- start 2024-04-26T21:47:20.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.676+0000 7fdfaa800640 1 -- start start 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfaa800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfa4146680 0x7fdfa40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfaa800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 0x7fdfa40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfaa800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa4146a60 con 0x7fdfa40d0e70 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfaa800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa406e970 con 0x7fdfa4146680 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfaa800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa40d17f0 con 0x7fdfa40b4b30 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa8a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdfa40d0e70 0x7fdfa4160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43164/0 (socket says 172.21.15.106:43164) 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa8a00640 1 -- 172.21.15.106:0/3254718173 learned_addr learned my addr 172.21.15.106:0/3254718173 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3254718173 >> v2:172.21.15.106:3300/0 conn(0x7fdfa4146680 0x7fdfa40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa9e00640 1 --2- 172.21.15.106:0/3254718173 >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 0x7fdfa40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa3e00640 1 -- 172.21.15.106:0/3254718173 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1348467510 0 0) 0x7fdfa4146a60 con 0x7fdfa40d0e70 2024-04-26T21:47:20.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.678+0000 7fdfa3e00640 1 -- 172.21.15.106:0/3254718173 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c003520 con 0x7fdfa40d0e70 2024-04-26T21:47:20.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.679+0000 7fdfa9400640 1 -- 172.21.15.106:0/3254718173 >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 msgr2=0x7fdfa40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:20.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.679+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3254718173 >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 0x7fdfa40d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.679+0000 7fdfa9400640 1 -- 172.21.15.106:0/3254718173 >> v1:172.21.15.106:6789/0 conn(0x7fdfa40d0e70 legacy=0x7fdfa4160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.679+0000 7fdfa9400640 1 -- 172.21.15.106:0/3254718173 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa406dd60 con 0x7fdfa4146680 2024-04-26T21:47:20.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.679+0000 7fdfa9e00640 1 --2- 172.21.15.106:0/3254718173 >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 0x7fdfa40d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:20.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.680+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3254718173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa4146680 0x7fdfa40b5990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdfa406f370 tx=0x7fdf9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe32a4d9e5030be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.680+0000 7fdfa3e00640 1 -- 172.21.15.106:0/3254718173 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98037070 con 0x7fdfa4146680 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.680+0000 7fdfa3e00640 1 -- 172.21.15.106:0/3254718173 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf98032440 con 0x7fdfa4146680 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.680+0000 7fdfa3e00640 1 -- 172.21.15.106:0/3254718173 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98032810 con 0x7fdfa4146680 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 -- 172.21.15.106:0/3254718173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa4146680 msgr2=0x7fdfa40b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3254718173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa4146680 0x7fdfa40b5990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdfa406f370 tx=0x7fdf9802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 -- 172.21.15.106:0/3254718173 shutdown_connections 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3254718173 >> v2:172.21.15.204:3300/0 conn(0x7fdfa40b4b30 0x7fdfa40d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3254718173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa4146680 0x7fdfa40b5990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 -- 172.21.15.106:0/3254718173 >> 172.21.15.106:0/3254718173 conn(0x7fdfa406c740 msgr2=0x7fdfa406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 -- 172.21.15.106:0/3254718173 shutdown_connections 2024-04-26T21:47:20.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.681+0000 7fdfaa800640 1 -- 172.21.15.106:0/3254718173 wait complete. 2024-04-26T21:47:20.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.682+0000 7fdfaa800640 1 Processor -- start 2024-04-26T21:47:20.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 -- start start 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfa40d0e70 0x7fdfa413d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfa4146680 0x7fdfa413da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa406e600 con 0x7fdfa40b4b30 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa406e320 con 0x7fdfa40d0e70 2024-04-26T21:47:20.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.683+0000 7fdfaa800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa40cd6a0 con 0x7fdfa4146680 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53538/0 (socket says 172.21.15.106:53538) 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 -- 172.21.15.106:0/3691207246 learned_addr learned my addr 172.21.15.106:0/3691207246 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa8a00640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfa40d0e70 0x7fdfa413d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 -- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfa4146680 msgr2=0x7fdfa413da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfa4146680 0x7fdfa413da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 -- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfa40d0e70 msgr2=0x7fdfa413d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfa40d0e70 0x7fdfa413d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 -- 172.21.15.106:0/3691207246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa413bae0 con 0x7fdfa40b4b30 2024-04-26T21:47:20.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.684+0000 7fdfa9400640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802a1e0 tx=0x7fdf980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.685+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98037040 con 0x7fdfa40b4b30 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.685+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf980304c0 con 0x7fdfa40b4b30 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.685+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98030890 con 0x7fdfa40b4b30 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.685+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa413bdc0 con 0x7fdfa40b4b30 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.685+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa40ccf60 con 0x7fdfa40b4b30 2024-04-26T21:47:20.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.686+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa40cae10 con 0x7fdfa40b4b30 2024-04-26T21:47:20.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.687+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf98044020 con 0x7fdfa40b4b30 2024-04-26T21:47:20.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.688+0000 7fdfa1600640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 0x7fdf8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.688+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdf980abec0 con 0x7fdfa40b4b30 2024-04-26T21:47:20.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.688+0000 7fdfa8a00640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 0x7fdf8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.689+0000 7fdfa8a00640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 0x7fdf8406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfa41456b0 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.693+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf9807a250 con 0x7fdfa40b4b30 2024-04-26T21:47:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.874+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfa413df40 con 0x7fdf8406c8f0 2024-04-26T21:47:20.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.887+0000 7fdfa1600640 1 -- 172.21.15.106:0/3691207246 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7fdfa413df40 con 0x7fdf8406c8f0 2024-04-26T21:47:20.891 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:20.892 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:47:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 msgr2=0x7fdf8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 0x7fdf8406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfa41456b0 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 msgr2=0x7fdfa4140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802a1e0 tx=0x7fdf980046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 shutdown_connections 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdf8406c8f0 0x7fdf8406edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfa4146680 0x7fdfa413da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfa40d0e70 0x7fdfa413d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 --2- 172.21.15.106:0/3691207246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfa40b4b30 0x7fdfa4140060 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.892+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 >> 172.21.15.106:0/3691207246 conn(0x7fdfa406c740 msgr2=0x7fdfa415d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.893+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 shutdown_connections 2024-04-26T21:47:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:20.893+0000 7fdfaa800640 1 -- 172.21.15.106:0/3691207246 wait complete. 2024-04-26T21:47:20.910 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T21:47:19.113548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220020,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754700,"statfs":{"total":773094113280,"available":772868812800,"internally_reserved":0,"allocated":3936256,"data_stored":2230199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.676889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469233+0000","last_change":"2024-04-26T21:47:14.469233+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469233+0000","last_undegraded":"2024-04-26T21:47:14.469233+0000","last_fullsized":"2024-04-26T21:47:14.469233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.473349+0000","last_change":"2024-04-26T21:47:14.473349+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.473349+0000","last_undegraded":"2024-04-26T21:47:14.473349+0000","last_fullsized":"2024-04-26T21:47:14.473349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466695+0000","last_change":"2024-04-26T21:47:14.466695+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466695+0000","last_undegraded":"2024-04-26T21:47:14.466695+0000","last_fullsized":"2024-04-26T21:47:14.466695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463852+0000","last_change":"2024-04-26T21:47:14.463852+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463852+0000","last_undegraded":"2024-04-26T21:47:14.463852+0000","last_fullsized":"2024-04-26T21:47:14.463852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.466007+0000","last_change":"2024-04-26T21:47:14.466007+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.466007+0000","last_undegraded":"2024-04-26T21:47:14.466007+0000","last_fullsized":"2024-04-26T21:47:14.466007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.471474+0000","last_change":"2024-04-26T21:47:14.471474+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.471474+0000","last_undegraded":"2024-04-26T21:47:14.471474+0000","last_fullsized":"2024-04-26T21:47:14.471474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452768+0000","last_change":"2024-04-26T21:47:14.452768+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452768+0000","last_undegraded":"2024-04-26T21:47:14.452768+0000","last_fullsized":"2024-04-26T21:47:14.452768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457237+0000","last_change":"2024-04-26T21:47:14.457237+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457237+0000","last_undegraded":"2024-04-26T21:47:14.457237+0000","last_fullsized":"2024-04-26T21:47:14.457237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453766+0000","last_change":"2024-04-26T21:47:14.453766+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453766+0000","last_undegraded":"2024-04-26T21:47:14.453766+0000","last_fullsized":"2024-04-26T21:47:14.453766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462059+0000","last_change":"2024-04-26T21:47:14.462059+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462059+0000","last_undegraded":"2024-04-26T21:47:14.462059+0000","last_fullsized":"2024-04-26T21:47:14.462059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453846+0000","last_change":"2024-04-26T21:47:14.453846+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453846+0000","last_undegraded":"2024-04-26T21:47:14.453846+0000","last_fullsized":"2024-04-26T21:47:14.453846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465401+0000","last_change":"2024-04-26T21:47:14.465401+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465401+0000","last_undegraded":"2024-04-26T21:47:14.465401+0000","last_fullsized":"2024-04-26T21:47:14.465401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459374+0000","last_change":"2024-04-26T21:47:14.459374+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459374+0000","last_undegraded":"2024-04-26T21:47:14.459374+0000","last_fullsized":"2024-04-26T21:47:14.459374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470943+0000","last_change":"2024-04-26T21:47:14.470943+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470943+0000","last_undegraded":"2024-04-26T21:47:14.470943+0000","last_fullsized":"2024-04-26T21:47:14.470943+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470934+0000","last_change":"2024-04-26T21:47:14.470934+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470934+0000","last_undegraded":"2024-04-26T21:47:14.470934+0000","last_fullsized":"2024-04-26T21:47:14.470934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449141+0000","last_change":"2024-04-26T21:47:14.449141+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449141+0000","last_undegraded":"2024-04-26T21:47:14.449141+0000","last_fullsized":"2024-04-26T21:47:14.449141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450358+0000","last_change":"2024-04-26T21:47:14.450358+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450358+0000","last_undegraded":"2024-04-26T21:47:14.450358+0000","last_fullsized":"2024-04-26T21:47:14.450358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470776+0000","last_change":"2024-04-26T21:47:14.470776+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470776+0000","last_undegraded":"2024-04-26T21:47:14.470776+0000","last_fullsized":"2024-04-26T21:47:14.470776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469285+0000","last_change":"2024-04-26T21:47:14.469285+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469285+0000","last_undegraded":"2024-04-26T21:47:14.469285+0000","last_fullsized":"2024-04-26T21:47:14.469285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457654+0000","last_change":"2024-04-26T21:47:14.457654+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457654+0000","last_undegraded":"2024-04-26T21:47:14.457654+0000","last_fullsized":"2024-04-26T21:47:14.457654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457377+0000","last_change":"2024-04-26T21:47:14.457377+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457377+0000","last_undegraded":"2024-04-26T21:47:14.457377+0000","last_fullsized":"2024-04-26T21:47:14.457377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452220+0000","last_change":"2024-04-26T21:47:14.452220+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452220+0000","last_undegraded":"2024-04-26T21:47:14.452220+0000","last_fullsized":"2024-04-26T21:47:14.452220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462364+0000","last_change":"2024-04-26T21:47:14.462364+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462364+0000","last_undegraded":"2024-04-26T21:47:14.462364+0000","last_fullsized":"2024-04-26T21:47:14.462364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.469923+0000","last_change":"2024-04-26T21:47:14.469923+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.469923+0000","last_undegraded":"2024-04-26T21:47:14.469923+0000","last_fullsized":"2024-04-26T21:47:14.469923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.460700+0000","last_change":"2024-04-26T21:47:14.460700+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.460700+0000","last_undegraded":"2024-04-26T21:47:14.460700+0000","last_fullsized":"2024-04-26T21:47:14.460700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468882+0000","last_change":"2024-04-26T21:47:14.468882+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468882+0000","last_undegraded":"2024-04-26T21:47:14.468882+0000","last_fullsized":"2024-04-26T21:47:14.468882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465607+0000","last_change":"2024-04-26T21:47:14.465607+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465607+0000","last_undegraded":"2024-04-26T21:47:14.465607+0000","last_fullsized":"2024-04-26T21:47:14.465607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463389+0000","last_change":"2024-04-26T21:47:14.463389+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463389+0000","last_undegraded":"2024-04-26T21:47:14.463389+0000","last_fullsized":"2024-04-26T21:47:14.463389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456256+0000","last_change":"2024-04-26T21:47:14.456256+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456256+0000","last_undegraded":"2024-04-26T21:47:14.456256+0000","last_fullsized":"2024-04-26T21:47:14.456256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450566+0000","last_change":"2024-04-26T21:47:14.450566+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450566+0000","last_undegraded":"2024-04-26T21:47:14.450566+0000","last_fullsized":"2024-04-26T21:47:14.450566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468593+0000","last_change":"2024-04-26T21:47:14.468593+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468593+0000","last_undegraded":"2024-04-26T21:47:14.468593+0000","last_fullsized":"2024-04-26T21:47:14.468593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456198+0000","last_change":"2024-04-26T21:47:14.456198+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456198+0000","last_undegraded":"2024-04-26T21:47:14.456198+0000","last_fullsized":"2024-04-26T21:47:14.456198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470298+0000","last_change":"2024-04-26T21:47:14.470298+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470298+0000","last_undegraded":"2024-04-26T21:47:14.470298+0000","last_fullsized":"2024-04-26T21:47:14.470298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488061+0000","last_change":"2024-04-26T21:47:14.483471+0000","last_active":"2024-04-26T21:47:14.488061+0000","last_peered":"2024-04-26T21:47:14.488061+0000","last_clean":"2024-04-26T21:47:14.488061+0000","last_became_active":"2024-04-26T21:47:14.483132+0000","last_became_peered":"2024-04-26T21:47:14.483132+0000","last_unstale":"2024-04-26T21:47:14.488061+0000","last_undegraded":"2024-04-26T21:47:14.488061+0000","last_fullsized":"2024-04-26T21:47:14.488061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468665+0000","last_change":"2024-04-26T21:47:14.468665+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468665+0000","last_undegraded":"2024-04-26T21:47:14.468665+0000","last_fullsized":"2024-04-26T21:47:14.468665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486611+0000","last_change":"2024-04-26T21:47:14.477793+0000","last_active":"2024-04-26T21:47:14.486611+0000","last_peered":"2024-04-26T21:47:14.486611+0000","last_clean":"2024-04-26T21:47:14.486611+0000","last_became_active":"2024-04-26T21:47:14.477460+0000","last_became_peered":"2024-04-26T21:47:14.477460+0000","last_unstale":"2024-04-26T21:47:14.486611+0000","last_undegraded":"2024-04-26T21:47:14.486611+0000","last_fullsized":"2024-04-26T21:47:14.486611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453644+0000","last_change":"2024-04-26T21:47:14.453644+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453644+0000","last_undegraded":"2024-04-26T21:47:14.453644+0000","last_fullsized":"2024-04-26T21:47:14.453644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490372+0000","last_change":"2024-04-26T21:47:14.484669+0000","last_active":"2024-04-26T21:47:14.490372+0000","last_peered":"2024-04-26T21:47:14.490372+0000","last_clean":"2024-04-26T21:47:14.490372+0000","last_became_active":"2024-04-26T21:47:14.483933+0000","last_became_peered":"2024-04-26T21:47:14.483933+0000","last_unstale":"2024-04-26T21:47:14.490372+0000","last_undegraded":"2024-04-26T21:47:14.490372+0000","last_fullsized":"2024-04-26T21:47:14.490372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463630+0000","last_change":"2024-04-26T21:47:14.463630+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463630+0000","last_undegraded":"2024-04-26T21:47:14.463630+0000","last_fullsized":"2024-04-26T21:47:14.463630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.469248+0000","last_change":"2024-04-26T21:47:14.462571+0000","last_active":"2024-04-26T21:47:14.469248+0000","last_peered":"2024-04-26T21:47:14.469248+0000","last_clean":"2024-04-26T21:47:14.469248+0000","last_became_active":"2024-04-26T21:47:14.461470+0000","last_became_peered":"2024-04-26T21:47:14.461470+0000","last_unstale":"2024-04-26T21:47:14.469248+0000","last_undegraded":"2024-04-26T21:47:14.469248+0000","last_fullsized":"2024-04-26T21:47:14.469248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.449275+0000","last_change":"2024-04-26T21:47:14.449275+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.449275+0000","last_undegraded":"2024-04-26T21:47:14.449275+0000","last_fullsized":"2024-04-26T21:47:14.449275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490646+0000","last_change":"2024-04-26T21:47:14.485718+0000","last_active":"2024-04-26T21:47:14.490646+0000","last_peered":"2024-04-26T21:47:14.490646+0000","last_clean":"2024-04-26T21:47:14.490646+0000","last_became_active":"2024-04-26T21:47:14.485458+0000","last_became_peered":"2024-04-26T21:47:14.485458+0000","last_unstale":"2024-04-26T21:47:14.490646+0000","last_undegraded":"2024-04-26T21:47:14.490646+0000","last_fullsized":"2024-04-26T21:47:14.490646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:12.839326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463772+0000","last_change":"2024-04-26T21:47:14.463772+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463772+0000","last_undegraded":"2024-04-26T21:47:14.463772+0000","last_fullsized":"2024-04-26T21:47:14.463772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.488020+0000","last_change":"2024-04-26T21:47:14.484537+0000","last_active":"2024-04-26T21:47:14.488020+0000","last_peered":"2024-04-26T21:47:14.488020+0000","last_clean":"2024-04-26T21:47:14.488020+0000","last_became_active":"2024-04-26T21:47:14.484218+0000","last_became_peered":"2024-04-26T21:47:14.484218+0000","last_unstale":"2024-04-26T21:47:14.488020+0000","last_undegraded":"2024-04-26T21:47:14.488020+0000","last_fullsized":"2024-04-26T21:47:14.488020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:23.713847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.452619+0000","last_change":"2024-04-26T21:47:14.452619+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.452619+0000","last_undegraded":"2024-04-26T21:47:14.452619+0000","last_fullsized":"2024-04-26T21:47:14.452619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490039+0000","last_change":"2024-04-26T21:47:14.483496+0000","last_active":"2024-04-26T21:47:14.490039+0000","last_peered":"2024-04-26T21:47:14.490039+0000","last_clean":"2024-04-26T21:47:14.490039+0000","last_became_active":"2024-04-26T21:47:14.483201+0000","last_became_peered":"2024-04-26T21:47:14.483201+0000","last_unstale":"2024-04-26T21:47:14.490039+0000","last_undegraded":"2024-04-26T21:47:14.490039+0000","last_fullsized":"2024-04-26T21:47:14.490039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463534+0000","last_change":"2024-04-26T21:47:14.463534+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463534+0000","last_undegraded":"2024-04-26T21:47:14.463534+0000","last_fullsized":"2024-04-26T21:47:14.463534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482982+0000","last_change":"2024-04-26T21:47:14.480135+0000","last_active":"2024-04-26T21:47:14.482982+0000","last_peered":"2024-04-26T21:47:14.482982+0000","last_clean":"2024-04-26T21:47:14.482982+0000","last_became_active":"2024-04-26T21:47:14.479762+0000","last_became_peered":"2024-04-26T21:47:14.479762+0000","last_unstale":"2024-04-26T21:47:14.482982+0000","last_undegraded":"2024-04-26T21:47:14.482982+0000","last_fullsized":"2024-04-26T21:47:14.482982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:37.394548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.463597+0000","last_change":"2024-04-26T21:47:14.463597+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.463597+0000","last_undegraded":"2024-04-26T21:47:14.463597+0000","last_fullsized":"2024-04-26T21:47:14.463597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490144+0000","last_change":"2024-04-26T21:47:14.484137+0000","last_active":"2024-04-26T21:47:14.490144+0000","last_peered":"2024-04-26T21:47:14.490144+0000","last_clean":"2024-04-26T21:47:14.490144+0000","last_became_active":"2024-04-26T21:47:14.483547+0000","last_became_peered":"2024-04-26T21:47:14.483547+0000","last_unstale":"2024-04-26T21:47:14.490144+0000","last_undegraded":"2024-04-26T21:47:14.490144+0000","last_fullsized":"2024-04-26T21:47:14.490144+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.476712+0000","last_change":"2024-04-26T21:47:14.476712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.476712+0000","last_undegraded":"2024-04-26T21:47:14.476712+0000","last_fullsized":"2024-04-26T21:47:14.476712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489763+0000","last_change":"2024-04-26T21:47:14.485542+0000","last_active":"2024-04-26T21:47:14.489763+0000","last_peered":"2024-04-26T21:47:14.489763+0000","last_clean":"2024-04-26T21:47:14.489763+0000","last_became_active":"2024-04-26T21:47:14.485231+0000","last_became_peered":"2024-04-26T21:47:14.485231+0000","last_unstale":"2024-04-26T21:47:14.489763+0000","last_undegraded":"2024-04-26T21:47:14.489763+0000","last_fullsized":"2024-04-26T21:47:14.489763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.475921+0000","last_change":"2024-04-26T21:47:14.475921+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.475921+0000","last_undegraded":"2024-04-26T21:47:14.475921+0000","last_fullsized":"2024-04-26T21:47:14.475921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.480096+0000","last_change":"2024-04-26T21:47:14.472755+0000","last_active":"2024-04-26T21:47:14.480096+0000","last_peered":"2024-04-26T21:47:14.480096+0000","last_clean":"2024-04-26T21:47:14.480096+0000","last_became_active":"2024-04-26T21:47:14.472547+0000","last_became_peered":"2024-04-26T21:47:14.472547+0000","last_unstale":"2024-04-26T21:47:14.480096+0000","last_undegraded":"2024-04-26T21:47:14.480096+0000","last_fullsized":"2024-04-26T21:47:14.480096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467306+0000","last_change":"2024-04-26T21:47:14.467306+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467306+0000","last_undegraded":"2024-04-26T21:47:14.467306+0000","last_fullsized":"2024-04-26T21:47:14.467306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493863+0000","last_change":"2024-04-26T21:47:14.487095+0000","last_active":"2024-04-26T21:47:14.493863+0000","last_peered":"2024-04-26T21:47:14.493863+0000","last_clean":"2024-04-26T21:47:14.493863+0000","last_became_active":"2024-04-26T21:47:14.486802+0000","last_became_peered":"2024-04-26T21:47:14.486802+0000","last_unstale":"2024-04-26T21:47:14.493863+0000","last_undegraded":"2024-04-26T21:47:14.493863+0000","last_fullsized":"2024-04-26T21:47:14.493863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.461282+0000","last_change":"2024-04-26T21:47:14.461282+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.461282+0000","last_undegraded":"2024-04-26T21:47:14.461282+0000","last_fullsized":"2024-04-26T21:47:14.461282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481227+0000","last_change":"2024-04-26T21:47:14.478529+0000","last_active":"2024-04-26T21:47:14.481227+0000","last_peered":"2024-04-26T21:47:14.481227+0000","last_clean":"2024-04-26T21:47:14.481227+0000","last_became_active":"2024-04-26T21:47:14.478281+0000","last_became_peered":"2024-04-26T21:47:14.478281+0000","last_unstale":"2024-04-26T21:47:14.481227+0000","last_undegraded":"2024-04-26T21:47:14.481227+0000","last_fullsized":"2024-04-26T21:47:14.481227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:02.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465187+0000","last_change":"2024-04-26T21:47:14.465187+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465187+0000","last_undegraded":"2024-04-26T21:47:14.465187+0000","last_fullsized":"2024-04-26T21:47:14.465187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481931+0000","last_change":"2024-04-26T21:47:14.480124+0000","last_active":"2024-04-26T21:47:14.481931+0000","last_peered":"2024-04-26T21:47:14.481931+0000","last_clean":"2024-04-26T21:47:14.481931+0000","last_became_active":"2024-04-26T21:47:14.479846+0000","last_became_peered":"2024-04-26T21:47:14.479846+0000","last_unstale":"2024-04-26T21:47:14.481931+0000","last_undegraded":"2024-04-26T21:47:14.481931+0000","last_fullsized":"2024-04-26T21:47:14.481931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.455199+0000","last_change":"2024-04-26T21:47:14.455199+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.455199+0000","last_undegraded":"2024-04-26T21:47:14.455199+0000","last_fullsized":"2024-04-26T21:47:14.455199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490380+0000","last_change":"2024-04-26T21:47:14.485004+0000","last_active":"2024-04-26T21:47:14.490380+0000","last_peered":"2024-04-26T21:47:14.490380+0000","last_clean":"2024-04-26T21:47:14.490380+0000","last_became_active":"2024-04-26T21:47:14.483355+0000","last_became_peered":"2024-04-26T21:47:14.483355+0000","last_unstale":"2024-04-26T21:47:14.490380+0000","last_undegraded":"2024-04-26T21:47:14.490380+0000","last_fullsized":"2024-04-26T21:47:14.490380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467945+0000","last_change":"2024-04-26T21:47:14.467945+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467945+0000","last_undegraded":"2024-04-26T21:47:14.467945+0000","last_fullsized":"2024-04-26T21:47:14.467945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487389+0000","last_change":"2024-04-26T21:47:14.482285+0000","last_active":"2024-04-26T21:47:14.487389+0000","last_peered":"2024-04-26T21:47:14.487389+0000","last_clean":"2024-04-26T21:47:14.487389+0000","last_became_active":"2024-04-26T21:47:14.481952+0000","last_became_peered":"2024-04-26T21:47:14.481952+0000","last_unstale":"2024-04-26T21:47:14.487389+0000","last_undegraded":"2024-04-26T21:47:14.487389+0000","last_fullsized":"2024-04-26T21:47:14.487389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.456504+0000","last_change":"2024-04-26T21:47:14.456504+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.456504+0000","last_undegraded":"2024-04-26T21:47:14.456504+0000","last_fullsized":"2024-04-26T21:47:14.456504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.493983+0000","last_change":"2024-04-26T21:47:14.485819+0000","last_active":"2024-04-26T21:47:14.493983+0000","last_peered":"2024-04-26T21:47:14.493983+0000","last_clean":"2024-04-26T21:47:14.493983+0000","last_became_active":"2024-04-26T21:47:14.485435+0000","last_became_peered":"2024-04-26T21:47:14.485435+0000","last_unstale":"2024-04-26T21:47:14.493983+0000","last_undegraded":"2024-04-26T21:47:14.493983+0000","last_fullsized":"2024-04-26T21:47:14.493983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:48.711514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465748+0000","last_change":"2024-04-26T21:47:14.465748+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465748+0000","last_undegraded":"2024-04-26T21:47:14.465748+0000","last_fullsized":"2024-04-26T21:47:14.465748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485819+0000","last_change":"2024-04-26T21:47:14.481662+0000","last_active":"2024-04-26T21:47:14.485819+0000","last_peered":"2024-04-26T21:47:14.485819+0000","last_clean":"2024-04-26T21:47:14.485819+0000","last_became_active":"2024-04-26T21:47:14.480582+0000","last_became_peered":"2024-04-26T21:47:14.480582+0000","last_unstale":"2024-04-26T21:47:14.485819+0000","last_undegraded":"2024-04-26T21:47:14.485819+0000","last_fullsized":"2024-04-26T21:47:14.485819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:00.441765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.462995+0000","last_change":"2024-04-26T21:47:14.462995+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.462995+0000","last_undegraded":"2024-04-26T21:47:14.462995+0000","last_fullsized":"2024-04-26T21:47:14.462995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478547+0000","last_change":"2024-04-26T21:47:14.475423+0000","last_active":"2024-04-26T21:47:14.478547+0000","last_peered":"2024-04-26T21:47:14.478547+0000","last_clean":"2024-04-26T21:47:14.478547+0000","last_became_active":"2024-04-26T21:47:14.475083+0000","last_became_peered":"2024-04-26T21:47:14.475083+0000","last_unstale":"2024-04-26T21:47:14.478547+0000","last_undegraded":"2024-04-26T21:47:14.478547+0000","last_fullsized":"2024-04-26T21:47:14.478547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459712+0000","last_change":"2024-04-26T21:47:14.459712+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459712+0000","last_undegraded":"2024-04-26T21:47:14.459712+0000","last_fullsized":"2024-04-26T21:47:14.459712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490682+0000","last_change":"2024-04-26T21:47:14.487648+0000","last_active":"2024-04-26T21:47:14.490682+0000","last_peered":"2024-04-26T21:47:14.490682+0000","last_clean":"2024-04-26T21:47:14.490682+0000","last_became_active":"2024-04-26T21:47:14.487356+0000","last_became_peered":"2024-04-26T21:47:14.487356+0000","last_unstale":"2024-04-26T21:47:14.490682+0000","last_undegraded":"2024-04-26T21:47:14.490682+0000","last_fullsized":"2024-04-26T21:47:14.490682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:02.916803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458838+0000","last_change":"2024-04-26T21:47:14.458838+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458838+0000","last_undegraded":"2024-04-26T21:47:14.458838+0000","last_fullsized":"2024-04-26T21:47:14.458838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486377+0000","last_change":"2024-04-26T21:47:14.478983+0000","last_active":"2024-04-26T21:47:14.486377+0000","last_peered":"2024-04-26T21:47:14.486377+0000","last_clean":"2024-04-26T21:47:14.486377+0000","last_became_active":"2024-04-26T21:47:14.478698+0000","last_became_peered":"2024-04-26T21:47:14.478698+0000","last_unstale":"2024-04-26T21:47:14.486377+0000","last_undegraded":"2024-04-26T21:47:14.486377+0000","last_fullsized":"2024-04-26T21:47:14.486377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.458086+0000","last_change":"2024-04-26T21:47:14.458086+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.458086+0000","last_undegraded":"2024-04-26T21:47:14.458086+0000","last_fullsized":"2024-04-26T21:47:14.458086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.487855+0000","last_change":"2024-04-26T21:47:14.482597+0000","last_active":"2024-04-26T21:47:14.487855+0000","last_peered":"2024-04-26T21:47:14.487855+0000","last_clean":"2024-04-26T21:47:14.487855+0000","last_became_active":"2024-04-26T21:47:14.482299+0000","last_became_peered":"2024-04-26T21:47:14.482299+0000","last_unstale":"2024-04-26T21:47:14.487855+0000","last_undegraded":"2024-04-26T21:47:14.487855+0000","last_fullsized":"2024-04-26T21:47:14.487855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:20.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457559+0000","last_change":"2024-04-26T21:47:14.457559+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457559+0000","last_undegraded":"2024-04-26T21:47:14.457559+0000","last_fullsized":"2024-04-26T21:47:14.457559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.478807+0000","last_change":"2024-04-26T21:47:14.475056+0000","last_active":"2024-04-26T21:47:14.478807+0000","last_peered":"2024-04-26T21:47:14.478807+0000","last_clean":"2024-04-26T21:47:14.478807+0000","last_became_active":"2024-04-26T21:47:14.474735+0000","last_became_peered":"2024-04-26T21:47:14.474735+0000","last_unstale":"2024-04-26T21:47:14.478807+0000","last_undegraded":"2024-04-26T21:47:14.478807+0000","last_fullsized":"2024-04-26T21:47:14.478807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:06.758168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.453724+0000","last_change":"2024-04-26T21:47:14.453724+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.453724+0000","last_undegraded":"2024-04-26T21:47:14.453724+0000","last_fullsized":"2024-04-26T21:47:14.453724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.482671+0000","last_change":"2024-04-26T21:47:14.478761+0000","last_active":"2024-04-26T21:47:14.482671+0000","last_peered":"2024-04-26T21:47:14.482671+0000","last_clean":"2024-04-26T21:47:14.482671+0000","last_became_active":"2024-04-26T21:47:14.477387+0000","last_became_peered":"2024-04-26T21:47:14.477387+0000","last_unstale":"2024-04-26T21:47:14.482671+0000","last_undegraded":"2024-04-26T21:47:14.482671+0000","last_fullsized":"2024-04-26T21:47:14.482671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.457583+0000","last_change":"2024-04-26T21:47:14.457583+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.457583+0000","last_undegraded":"2024-04-26T21:47:14.457583+0000","last_fullsized":"2024-04-26T21:47:14.457583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481460+0000","last_change":"2024-04-26T21:47:14.476244+0000","last_active":"2024-04-26T21:47:14.481460+0000","last_peered":"2024-04-26T21:47:14.481460+0000","last_clean":"2024-04-26T21:47:14.481460+0000","last_became_active":"2024-04-26T21:47:14.475929+0000","last_became_peered":"2024-04-26T21:47:14.475929+0000","last_unstale":"2024-04-26T21:47:14.481460+0000","last_undegraded":"2024-04-26T21:47:14.481460+0000","last_fullsized":"2024-04-26T21:47:14.481460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.465106+0000","last_change":"2024-04-26T21:47:14.465106+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.465106+0000","last_undegraded":"2024-04-26T21:47:14.465106+0000","last_fullsized":"2024-04-26T21:47:14.465106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.486639+0000","last_change":"2024-04-26T21:47:14.479190+0000","last_active":"2024-04-26T21:47:14.486639+0000","last_peered":"2024-04-26T21:47:14.486639+0000","last_clean":"2024-04-26T21:47:14.486639+0000","last_became_active":"2024-04-26T21:47:14.478861+0000","last_became_peered":"2024-04-26T21:47:14.478861+0000","last_unstale":"2024-04-26T21:47:14.486639+0000","last_undegraded":"2024-04-26T21:47:14.486639+0000","last_fullsized":"2024-04-26T21:47:14.486639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.468034+0000","last_change":"2024-04-26T21:47:14.468034+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.468034+0000","last_undegraded":"2024-04-26T21:47:14.468034+0000","last_fullsized":"2024-04-26T21:47:14.468034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.474178+0000","last_change":"2024-04-26T21:47:14.465459+0000","last_active":"2024-04-26T21:47:14.474178+0000","last_peered":"2024-04-26T21:47:14.474178+0000","last_clean":"2024-04-26T21:47:14.474178+0000","last_became_active":"2024-04-26T21:47:14.464667+0000","last_became_peered":"2024-04-26T21:47:14.464667+0000","last_unstale":"2024-04-26T21:47:14.474178+0000","last_undegraded":"2024-04-26T21:47:14.474178+0000","last_fullsized":"2024-04-26T21:47:14.474178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470788+0000","last_change":"2024-04-26T21:47:14.470788+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470788+0000","last_undegraded":"2024-04-26T21:47:14.470788+0000","last_fullsized":"2024-04-26T21:47:14.470788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.489741+0000","last_change":"2024-04-26T21:47:14.481643+0000","last_active":"2024-04-26T21:47:14.489741+0000","last_peered":"2024-04-26T21:47:14.489741+0000","last_clean":"2024-04-26T21:47:14.489741+0000","last_became_active":"2024-04-26T21:47:14.481119+0000","last_became_peered":"2024-04-26T21:47:14.481119+0000","last_unstale":"2024-04-26T21:47:14.489741+0000","last_undegraded":"2024-04-26T21:47:14.489741+0000","last_fullsized":"2024-04-26T21:47:14.489741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:54.749000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.470626+0000","last_change":"2024-04-26T21:47:14.470626+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.470626+0000","last_undegraded":"2024-04-26T21:47:14.470626+0000","last_fullsized":"2024-04-26T21:47:14.470626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481555+0000","last_change":"2024-04-26T21:47:14.475520+0000","last_active":"2024-04-26T21:47:14.481555+0000","last_peered":"2024-04-26T21:47:14.481555+0000","last_clean":"2024-04-26T21:47:14.481555+0000","last_became_active":"2024-04-26T21:47:14.475208+0000","last_became_peered":"2024-04-26T21:47:14.475208+0000","last_unstale":"2024-04-26T21:47:14.481555+0000","last_undegraded":"2024-04-26T21:47:14.481555+0000","last_fullsized":"2024-04-26T21:47:14.481555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.467031+0000","last_change":"2024-04-26T21:47:14.467031+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.467031+0000","last_undegraded":"2024-04-26T21:47:14.467031+0000","last_fullsized":"2024-04-26T21:47:14.467031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.481067+0000","last_change":"2024-04-26T21:47:14.477761+0000","last_active":"2024-04-26T21:47:14.481067+0000","last_peered":"2024-04-26T21:47:14.481067+0000","last_clean":"2024-04-26T21:47:14.481067+0000","last_became_active":"2024-04-26T21:47:14.477292+0000","last_became_peered":"2024-04-26T21:47:14.477292+0000","last_unstale":"2024-04-26T21:47:14.481067+0000","last_undegraded":"2024-04-26T21:47:14.481067+0000","last_fullsized":"2024-04-26T21:47:14.481067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:49.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.459408+0000","last_change":"2024-04-26T21:47:14.459408+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.459408+0000","last_undegraded":"2024-04-26T21:47:14.459408+0000","last_fullsized":"2024-04-26T21:47:14.459408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.490408+0000","last_change":"2024-04-26T21:47:14.485177+0000","last_active":"2024-04-26T21:47:14.490408+0000","last_peered":"2024-04-26T21:47:14.490408+0000","last_clean":"2024-04-26T21:47:14.490408+0000","last_became_active":"2024-04-26T21:47:14.483874+0000","last_became_peered":"2024-04-26T21:47:14.483874+0000","last_unstale":"2024-04-26T21:47:14.490408+0000","last_undegraded":"2024-04-26T21:47:14.490408+0000","last_fullsized":"2024-04-26T21:47:14.490408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T21:47:14.450014+0000","last_change":"2024-04-26T21:47:14.450014+0000","last_active":"2024-04-26T21:47:14.434114+0000","last_peered":"2024-04-26T21:47:14.434114+0000","last_clean":"2024-04-26T21:47:14.434114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:14.450014+0000","last_undegraded":"2024-04-26T21:47:14.450014+0000","last_fullsized":"2024-04-26T21:47:14.450014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.485168+0000","last_change":"2024-04-26T21:47:14.480541+0000","last_active":"2024-04-26T21:47:14.485168+0000","last_peered":"2024-04-26T21:47:14.485168+0000","last_clean":"2024-04-26T21:47:14.485168+0000","last_became_active":"2024-04-26T21:47:14.480243+0000","last_became_peered":"2024-04-26T21:47:14.480243+0000","last_unstale":"2024-04-26T21:47:14.485168+0000","last_undegraded":"2024-04-26T21:47:14.485168+0000","last_fullsized":"2024-04-26T21:47:14.485168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:09.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T21:47:14.459791+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:14.459791+0000","last_peered":"2024-04-26T21:47:14.459791+0000","last_clean":"2024-04-26T21:47:14.459791+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:14.459791+0000","last_undegraded":"2024-04-26T21:47:14.459791+0000","last_fullsized":"2024-04-26T21:47:14.459791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:20.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-26T21:47:20.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.469233+0000', 'last_change': '2024-04-26T21:47:14.469233+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.469233+0000', 'last_undegraded': '2024-04-26T21:47:14.469233+0000', 'last_fullsized': '2024-04-26T21:47:14.469233+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.473349+0000', 'last_change': '2024-04-26T21:47:14.473349+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.473349+0000', 'last_undegraded': '2024-04-26T21:47:14.473349+0000', 'last_fullsized': '2024-04-26T21:47:14.473349+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.466695+0000', 'last_change': '2024-04-26T21:47:14.466695+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.466695+0000', 'last_undegraded': '2024-04-26T21:47:14.466695+0000', 'last_fullsized': '2024-04-26T21:47:14.466695+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463852+0000', 'last_change': '2024-04-26T21:47:14.463852+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463852+0000', 'last_undegraded': '2024-04-26T21:47:14.463852+0000', 'last_fullsized': '2024-04-26T21:47:14.463852+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.466007+0000', 'last_change': '2024-04-26T21:47:14.466007+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.466007+0000', 'last_undegraded': '2024-04-26T21:47:14.466007+0000', 'last_fullsized': '2024-04-26T21:47:14.466007+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.471474+0000', 'last_change': '2024-04-26T21:47:14.471474+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.471474+0000', 'last_undegraded': '2024-04-26T21:47:14.471474+0000', 'last_fullsized': '2024-04-26T21:47:14.471474+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.452768+0000', 'last_change': '2024-04-26T21:47:14.452768+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.452768+0000', 'last_undegraded': '2024-04-26T21:47:14.452768+0000', 'last_fullsized': '2024-04-26T21:47:14.452768+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.457237+0000', 'last_change': '2024-04-26T21:47:14.457237+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.457237+0000', 'last_undegraded': '2024-04-26T21:47:14.457237+0000', 'last_fullsized': '2024-04-26T21:47:14.457237+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.453766+0000', 'last_change': '2024-04-26T21:47:14.453766+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.453766+0000', 'last_undegraded': '2024-04-26T21:47:14.453766+0000', 'last_fullsized': '2024-04-26T21:47:14.453766+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.462059+0000', 'last_change': '2024-04-26T21:47:14.462059+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.462059+0000', 'last_undegraded': '2024-04-26T21:47:14.462059+0000', 'last_fullsized': '2024-04-26T21:47:14.462059+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.453846+0000', 'last_change': '2024-04-26T21:47:14.453846+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.453846+0000', 'last_undegraded': '2024-04-26T21:47:14.453846+0000', 'last_fullsized': '2024-04-26T21:47:14.453846+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.465401+0000', 'last_change': '2024-04-26T21:47:14.465401+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.465401+0000', 'last_undegraded': '2024-04-26T21:47:14.465401+0000', 'last_fullsized': '2024-04-26T21:47:14.465401+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.459374+0000', 'last_change': '2024-04-26T21:47:14.459374+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.459374+0000', 'last_undegraded': '2024-04-26T21:47:14.459374+0000', 'last_fullsized': '2024-04-26T21:47:14.459374+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470943+0000', 'last_change': '2024-04-26T21:47:14.470943+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470943+0000', 'last_undegraded': '2024-04-26T21:47:14.470943+0000', 'last_fullsized': '2024-04-26T21:47:14.470943+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470934+0000', 'last_change': '2024-04-26T21:47:14.470934+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470934+0000', 'last_undegraded': '2024-04-26T21:47:14.470934+0000', 'last_fullsized': '2024-04-26T21:47:14.470934+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-26T21:47:20.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.449141+0000', 'last_change': '2024-04-26T21:47:14.449141+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.449141+0000', 'last_undegraded': '2024-04-26T21:47:14.449141+0000', 'last_fullsized': '2024-04-26T21:47:14.449141+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.450358+0000', 'last_change': '2024-04-26T21:47:14.450358+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.450358+0000', 'last_undegraded': '2024-04-26T21:47:14.450358+0000', 'last_fullsized': '2024-04-26T21:47:14.450358+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470776+0000', 'last_change': '2024-04-26T21:47:14.470776+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470776+0000', 'last_undegraded': '2024-04-26T21:47:14.470776+0000', 'last_fullsized': '2024-04-26T21:47:14.470776+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.469285+0000', 'last_change': '2024-04-26T21:47:14.469285+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.469285+0000', 'last_undegraded': '2024-04-26T21:47:14.469285+0000', 'last_fullsized': '2024-04-26T21:47:14.469285+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.457654+0000', 'last_change': '2024-04-26T21:47:14.457654+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.457654+0000', 'last_undegraded': '2024-04-26T21:47:14.457654+0000', 'last_fullsized': '2024-04-26T21:47:14.457654+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.457377+0000', 'last_change': '2024-04-26T21:47:14.457377+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.457377+0000', 'last_undegraded': '2024-04-26T21:47:14.457377+0000', 'last_fullsized': '2024-04-26T21:47:14.457377+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.452220+0000', 'last_change': '2024-04-26T21:47:14.452220+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.452220+0000', 'last_undegraded': '2024-04-26T21:47:14.452220+0000', 'last_fullsized': '2024-04-26T21:47:14.452220+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.462364+0000', 'last_change': '2024-04-26T21:47:14.462364+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.462364+0000', 'last_undegraded': '2024-04-26T21:47:14.462364+0000', 'last_fullsized': '2024-04-26T21:47:14.462364+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.469923+0000', 'last_change': '2024-04-26T21:47:14.469923+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.469923+0000', 'last_undegraded': '2024-04-26T21:47:14.469923+0000', 'last_fullsized': '2024-04-26T21:47:14.469923+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.460700+0000', 'last_change': '2024-04-26T21:47:14.460700+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.460700+0000', 'last_undegraded': '2024-04-26T21:47:14.460700+0000', 'last_fullsized': '2024-04-26T21:47:14.460700+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.468882+0000', 'last_change': '2024-04-26T21:47:14.468882+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.468882+0000', 'last_undegraded': '2024-04-26T21:47:14.468882+0000', 'last_fullsized': '2024-04-26T21:47:14.468882+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.465607+0000', 'last_change': '2024-04-26T21:47:14.465607+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.465607+0000', 'last_undegraded': '2024-04-26T21:47:14.465607+0000', 'last_fullsized': '2024-04-26T21:47:14.465607+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463389+0000', 'last_change': '2024-04-26T21:47:14.463389+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463389+0000', 'last_undegraded': '2024-04-26T21:47:14.463389+0000', 'last_fullsized': '2024-04-26T21:47:14.463389+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.456256+0000', 'last_change': '2024-04-26T21:47:14.456256+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.456256+0000', 'last_undegraded': '2024-04-26T21:47:14.456256+0000', 'last_fullsized': '2024-04-26T21:47:14.456256+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.450566+0000', 'last_change': '2024-04-26T21:47:14.450566+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.450566+0000', 'last_undegraded': '2024-04-26T21:47:14.450566+0000', 'last_fullsized': '2024-04-26T21:47:14.450566+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.468593+0000', 'last_change': '2024-04-26T21:47:14.468593+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.468593+0000', 'last_undegraded': '2024-04-26T21:47:14.468593+0000', 'last_fullsized': '2024-04-26T21:47:14.468593+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:47:20.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.456198+0000', 'last_change': '2024-04-26T21:47:14.456198+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.456198+0000', 'last_undegraded': '2024-04-26T21:47:14.456198+0000', 'last_fullsized': '2024-04-26T21:47:14.456198+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470298+0000', 'last_change': '2024-04-26T21:47:14.470298+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470298+0000', 'last_undegraded': '2024-04-26T21:47:14.470298+0000', 'last_fullsized': '2024-04-26T21:47:14.470298+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.468665+0000', 'last_change': '2024-04-26T21:47:14.468665+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.468665+0000', 'last_undegraded': '2024-04-26T21:47:14.468665+0000', 'last_fullsized': '2024-04-26T21:47:14.468665+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.453644+0000', 'last_change': '2024-04-26T21:47:14.453644+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.453644+0000', 'last_undegraded': '2024-04-26T21:47:14.453644+0000', 'last_fullsized': '2024-04-26T21:47:14.453644+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463630+0000', 'last_change': '2024-04-26T21:47:14.463630+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463630+0000', 'last_undegraded': '2024-04-26T21:47:14.463630+0000', 'last_fullsized': '2024-04-26T21:47:14.463630+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-26T21:47:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.449275+0000', 'last_change': '2024-04-26T21:47:14.449275+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.449275+0000', 'last_undegraded': '2024-04-26T21:47:14.449275+0000', 'last_fullsized': '2024-04-26T21:47:14.449275+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463772+0000', 'last_change': '2024-04-26T21:47:14.463772+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463772+0000', 'last_undegraded': '2024-04-26T21:47:14.463772+0000', 'last_fullsized': '2024-04-26T21:47:14.463772+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.452619+0000', 'last_change': '2024-04-26T21:47:14.452619+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.452619+0000', 'last_undegraded': '2024-04-26T21:47:14.452619+0000', 'last_fullsized': '2024-04-26T21:47:14.452619+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463534+0000', 'last_change': '2024-04-26T21:47:14.463534+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463534+0000', 'last_undegraded': '2024-04-26T21:47:14.463534+0000', 'last_fullsized': '2024-04-26T21:47:14.463534+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.463597+0000', 'last_change': '2024-04-26T21:47:14.463597+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.463597+0000', 'last_undegraded': '2024-04-26T21:47:14.463597+0000', 'last_fullsized': '2024-04-26T21:47:14.463597+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.476712+0000', 'last_change': '2024-04-26T21:47:14.476712+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.476712+0000', 'last_undegraded': '2024-04-26T21:47:14.476712+0000', 'last_fullsized': '2024-04-26T21:47:14.476712+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.475921+0000', 'last_change': '2024-04-26T21:47:14.475921+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.475921+0000', 'last_undegraded': '2024-04-26T21:47:14.475921+0000', 'last_fullsized': '2024-04-26T21:47:14.475921+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.467306+0000', 'last_change': '2024-04-26T21:47:14.467306+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.467306+0000', 'last_undegraded': '2024-04-26T21:47:14.467306+0000', 'last_fullsized': '2024-04-26T21:47:14.467306+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.461282+0000', 'last_change': '2024-04-26T21:47:14.461282+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.461282+0000', 'last_undegraded': '2024-04-26T21:47:14.461282+0000', 'last_fullsized': '2024-04-26T21:47:14.461282+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.465187+0000', 'last_change': '2024-04-26T21:47:14.465187+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.465187+0000', 'last_undegraded': '2024-04-26T21:47:14.465187+0000', 'last_fullsized': '2024-04-26T21:47:14.465187+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.455199+0000', 'last_change': '2024-04-26T21:47:14.455199+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.455199+0000', 'last_undegraded': '2024-04-26T21:47:14.455199+0000', 'last_fullsized': '2024-04-26T21:47:14.455199+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.467945+0000', 'last_change': '2024-04-26T21:47:14.467945+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.467945+0000', 'last_undegraded': '2024-04-26T21:47:14.467945+0000', 'last_fullsized': '2024-04-26T21:47:14.467945+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.456504+0000', 'last_change': '2024-04-26T21:47:14.456504+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.456504+0000', 'last_undegraded': '2024-04-26T21:47:14.456504+0000', 'last_fullsized': '2024-04-26T21:47:14.456504+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.465748+0000', 'last_change': '2024-04-26T21:47:14.465748+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.465748+0000', 'last_undegraded': '2024-04-26T21:47:14.465748+0000', 'last_fullsized': '2024-04-26T21:47:14.465748+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.462995+0000', 'last_change': '2024-04-26T21:47:14.462995+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.462995+0000', 'last_undegraded': '2024-04-26T21:47:14.462995+0000', 'last_fullsized': '2024-04-26T21:47:14.462995+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.459712+0000', 'last_change': '2024-04-26T21:47:14.459712+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.459712+0000', 'last_undegraded': '2024-04-26T21:47:14.459712+0000', 'last_fullsized': '2024-04-26T21:47:14.459712+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.458838+0000', 'last_change': '2024-04-26T21:47:14.458838+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.458838+0000', 'last_undegraded': '2024-04-26T21:47:14.458838+0000', 'last_fullsized': '2024-04-26T21:47:14.458838+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.458086+0000', 'last_change': '2024-04-26T21:47:14.458086+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.458086+0000', 'last_undegraded': '2024-04-26T21:47:14.458086+0000', 'last_fullsized': '2024-04-26T21:47:14.458086+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.457559+0000', 'last_change': '2024-04-26T21:47:14.457559+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.457559+0000', 'last_undegraded': '2024-04-26T21:47:14.457559+0000', 'last_fullsized': '2024-04-26T21:47:14.457559+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.453724+0000', 'last_change': '2024-04-26T21:47:14.453724+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.453724+0000', 'last_undegraded': '2024-04-26T21:47:14.453724+0000', 'last_fullsized': '2024-04-26T21:47:14.453724+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.457583+0000', 'last_change': '2024-04-26T21:47:14.457583+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.457583+0000', 'last_undegraded': '2024-04-26T21:47:14.457583+0000', 'last_fullsized': '2024-04-26T21:47:14.457583+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.465106+0000', 'last_change': '2024-04-26T21:47:14.465106+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.465106+0000', 'last_undegraded': '2024-04-26T21:47:14.465106+0000', 'last_fullsized': '2024-04-26T21:47:14.465106+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-26T21:47:20.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.468034+0000', 'last_change': '2024-04-26T21:47:14.468034+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.468034+0000', 'last_undegraded': '2024-04-26T21:47:14.468034+0000', 'last_fullsized': '2024-04-26T21:47:14.468034+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470788+0000', 'last_change': '2024-04-26T21:47:14.470788+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470788+0000', 'last_undegraded': '2024-04-26T21:47:14.470788+0000', 'last_fullsized': '2024-04-26T21:47:14.470788+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.470626+0000', 'last_change': '2024-04-26T21:47:14.470626+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.470626+0000', 'last_undegraded': '2024-04-26T21:47:14.470626+0000', 'last_fullsized': '2024-04-26T21:47:14.470626+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.467031+0000', 'last_change': '2024-04-26T21:47:14.467031+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.467031+0000', 'last_undegraded': '2024-04-26T21:47:14.467031+0000', 'last_fullsized': '2024-04-26T21:47:14.467031+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.459408+0000', 'last_change': '2024-04-26T21:47:14.459408+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.459408+0000', 'last_undegraded': '2024-04-26T21:47:14.459408+0000', 'last_fullsized': '2024-04-26T21:47:14.459408+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:47:20.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-26T21:47:20.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:47:14.450014+0000', 'last_change': '2024-04-26T21:47:14.450014+0000', 'last_active': '2024-04-26T21:47:14.434114+0000', 'last_peered': '2024-04-26T21:47:14.434114+0000', 'last_clean': '2024-04-26T21:47:14.434114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:47:14.450014+0000', 'last_undegraded': '2024-04-26T21:47:14.450014+0000', 'last_fullsized': '2024-04-26T21:47:14.450014+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'last_clean_scrub_stamp': '2024-04-26T21:47:14.434114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:47:20.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:47:21.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.143+0000 7f3491600640 1 Processor -- start 2024-04-26T21:47:21.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.144+0000 7f3491600640 1 -- start start 2024-04-26T21:47:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f3491600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f348c0d3190 0x7f348c169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f3491600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348c15f5c0 con 0x7f348c15f1e0 2024-04-26T21:47:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f3491600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348c0b8670 con 0x7f348c0d5b60 2024-04-26T21:47:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f3491600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f348c0b8950 con 0x7f348c0d3190 2024-04-26T21:47:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f348be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f348c15f1e0 0x7f348c0b7d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43172/0 (socket says 172.21.15.106:43172) 2024-04-26T21:47:21.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.145+0000 7f348be00640 1 -- 172.21.15.106:0/750633044 learned_addr learned my addr 172.21.15.106:0/750633044 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f3490c00640 1 --2- 172.21.15.106:0/750633044 >> v2:172.21.15.204:3300/0 conn(0x7f348c0d3190 0x7f348c169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141046228 0 0) 0x7f348c15f5c0 con 0x7f348c15f1e0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f346c003520 con 0x7f348c15f1e0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1767084098 0 0) 0x7f346c003520 con 0x7f348c15f1e0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 >> v2:172.21.15.204:3300/0 conn(0x7f348c0d3190 msgr2=0x7f348c169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 --2- 172.21.15.106:0/750633044 >> v2:172.21.15.204:3300/0 conn(0x7f348c0d3190 0x7f348c169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 >> v1:172.21.15.204:6789/0 conn(0x7f348c0d5b60 legacy=0x7f348c0d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c0b8c30 con 0x7f348c15f1e0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f347c001670 con 0x7f348c15f1e0 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 >> v1:172.21.15.106:6789/0 conn(0x7f348c15f1e0 legacy=0x7f348c0b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f346c0051c0 0x7f346c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0b8950 con 0x7f346c0039b0 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f346c00c060 con 0x7f346c004800 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.146+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f346c00c260 con 0x7f346c0051c0 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348b400640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 -- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f346c0051c0 msgr2=0x7f346c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f346c0051c0 0x7f346c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 -- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c004800 msgr2=0x7f346c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f348be00640 1 -- 172.21.15.106:0/750633044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c003520 con 0x7f346c0039b0 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.147+0000 7f3490c00640 1 -- 172.21.15.106:0/750633044 reap_dead start 2024-04-26T21:47:21.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.148+0000 7f348be00640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f347c00c210 tx=0x7f346c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78626b292974020 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.148+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c007d30 con 0x7f346c0039b0 2024-04-26T21:47:21.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.148+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c0058d0 con 0x7f346c0039b0 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.148+0000 7f348aa00640 1 -- 172.21.15.106:0/750633044 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c005d20 con 0x7f346c0039b0 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 -- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 msgr2=0x7f346c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f347c00c210 tx=0x7f346c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 -- 172.21.15.106:0/750633044 shutdown_connections 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 --2- 172.21.15.106:0/750633044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 -- 172.21.15.106:0/750633044 >> 172.21.15.106:0/750633044 conn(0x7f348c09e2d0 msgr2=0x7f348c0d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 -- 172.21.15.106:0/750633044 shutdown_connections 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.149+0000 7f3491600640 1 -- 172.21.15.106:0/750633044 wait complete. 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.151+0000 7f3491600640 1 Processor -- start 2024-04-26T21:47:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.151+0000 7f3491600640 1 -- start start 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c0039b0 0x7f348c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f348c0cdd30 0x7f348c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0d2a80 con 0x7f348c154360 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0b91f0 con 0x7f346c0039b0 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3491600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f348c0d2c00 con 0x7f348c0cdd30 2024-04-26T21:47:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3490c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3490c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53556/0 (socket says 172.21.15.106:53556) 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3490c00640 1 -- 172.21.15.106:0/1792064140 learned_addr learned my addr 172.21.15.106:0/1792064140 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f348be00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c0039b0 0x7f348c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3490c00640 1 -- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f348c0cdd30 msgr2=0x7f348c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.152+0000 7f3490c00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f348c0cdd30 0x7f348c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3490c00640 1 -- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c0039b0 msgr2=0x7f348c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3490c00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c0039b0 0x7f348c15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3490c00640 1 -- 172.21.15.106:0/1792064140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c158130 con 0x7f348c154360 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3490c00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f348c08e890 tx=0x7f348400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348400c6b0 con 0x7f348c154360 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348401a070 con 0x7f348c154360 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c0c8b60 con 0x7f348c154360 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348400cad0 con 0x7f348c154360 2024-04-26T21:47:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.153+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c0c9120 con 0x7f348c154360 2024-04-26T21:47:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.154+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0693c0 con 0x7f348c154360 2024-04-26T21:47:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.155+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f348400cc70 con 0x7f348c154360 2024-04-26T21:47:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.156+0000 7f3483e00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 0x7f346806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.156+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f348408b620 con 0x7f348c154360 2024-04-26T21:47:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.156+0000 7f348be00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 0x7f346806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.157+0000 7f348be00640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 0x7f346806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f347c033000 tx=0x7f347c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.160+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34840599b0 con 0x7f348c154360 2024-04-26T21:47:21.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.436+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f348c06d050 con 0x7f348c154360 2024-04-26T21:47:21.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.438+0000 7f3483e00640 1 -- 172.21.15.106:0/1792064140 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f34840592b0 con 0x7f348c154360 2024-04-26T21:47:21.438 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:21.438 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714167944,"num_in_osds":8,"osd_in_since":1714167935,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225300480,"bytes_avail":772868812800,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":34541}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-26T21:47:03.109382+0000","services":{}},"progress_events":{}} 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.440+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 msgr2=0x7f346806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.440+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 0x7f346806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f347c033000 tx=0x7f347c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.440+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 msgr2=0x7f348c1547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.440+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f348c08e890 tx=0x7f348400ec00 comp rx=0 tx=0).stop 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.440+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 shutdown_connections 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348c154360 0x7f348c1547e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f346806c8f0 0x7f346806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f348c0cdd30 0x7f348c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 --2- 172.21.15.106:0/1792064140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f346c0039b0 0x7f348c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 >> 172.21.15.106:0/1792064140 conn(0x7f348c09e2d0 msgr2=0x7f348c0ba490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 shutdown_connections 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.441+0000 7f3491600640 1 -- 172.21.15.106:0/1792064140 wait complete. 2024-04-26T21:47:21.459 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:47:21.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:21.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.637+0000 7f1b9f000640 1 Processor -- start 2024-04-26T21:47:21.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.637+0000 7f1b9f000640 1 -- start start 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.638+0000 7f1b9f000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b980d0e70 0x7f1b980d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9f000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b98146a60 con 0x7f1b980b3b20 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9f000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9806dd60 con 0x7f1b98146680 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9f000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9806e970 con 0x7f1b980d0e70 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9e600640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1b980b3b20 0x7f1b980b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43188/0 (socket says 172.21.15.106:43188) 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9e600640 1 -- 172.21.15.106:0/1151433092 learned_addr learned my addr 172.21.15.106:0/1151433092 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9d200640 1 --2- 172.21.15.106:0/1151433092 >> v2:172.21.15.106:3300/0 conn(0x7f1b980d0e70 0x7f1b980d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9d200640 1 -- 172.21.15.106:0/1151433092 >> v1:172.21.15.204:6789/0 conn(0x7f1b98146680 legacy=0x7f1b980b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9d200640 1 -- 172.21.15.106:0/1151433092 >> v1:172.21.15.106:6789/0 conn(0x7f1b980b3b20 legacy=0x7f1b980b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b97e00640 1 -- 172.21.15.106:0/1151433092 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058876469 0 0) 0x7f1b98146a60 con 0x7f1b980b3b20 2024-04-26T21:47:21.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.639+0000 7f1b9d200640 1 -- 172.21.15.106:0/1151433092 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9806e040 con 0x7f1b980d0e70 2024-04-26T21:47:21.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.640+0000 7f1b9d200640 1 --2- 172.21.15.106:0/1151433092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980d0e70 0x7f1b980d12b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b840097a0 tx=0x7f1b8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec570622d96ae41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b97e00640 1 -- 172.21.15.106:0/1151433092 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84037070 con 0x7f1b980d0e70 2024-04-26T21:47:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b97e00640 1 -- 172.21.15.106:0/1151433092 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b84032440 con 0x7f1b980d0e70 2024-04-26T21:47:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b97e00640 1 -- 172.21.15.106:0/1151433092 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84032810 con 0x7f1b980d0e70 2024-04-26T21:47:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 -- 172.21.15.106:0/1151433092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980d0e70 msgr2=0x7f1b980d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 --2- 172.21.15.106:0/1151433092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980d0e70 0x7f1b980d12b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b840097a0 tx=0x7f1b8402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 -- 172.21.15.106:0/1151433092 shutdown_connections 2024-04-26T21:47:21.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 --2- 172.21.15.106:0/1151433092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980d0e70 0x7f1b980d12b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 -- 172.21.15.106:0/1151433092 >> 172.21.15.106:0/1151433092 conn(0x7f1b9806c740 msgr2=0x7f1b9806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 -- 172.21.15.106:0/1151433092 shutdown_connections 2024-04-26T21:47:21.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.641+0000 7f1b9f000640 1 -- 172.21.15.106:0/1151433092 wait complete. 2024-04-26T21:47:21.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.643+0000 7f1b9f000640 1 Processor -- start 2024-04-26T21:47:21.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.643+0000 7f1b9f000640 1 -- start start 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b980d0e70 0x7f1b98143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b98146680 0x7f1b9813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980d17f0 con 0x7f1b980b3b20 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980b60a0 con 0x7f1b98146680 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9806e600 con 0x7f1b980d0e70 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53574/0 (socket says 172.21.15.106:53574) 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.644+0000 7f1b9dc00640 1 -- 172.21.15.106:0/4151196292 learned_addr learned my addr 172.21.15.106:0/4151196292 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 -- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b980d0e70 msgr2=0x7f1b98143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b980d0e70 0x7f1b98143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 -- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b98146680 msgr2=0x7f1b9813fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b98146680 0x7f1b9813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 -- 172.21.15.106:0/4151196292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b98142540 con 0x7f1b980b3b20 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9dc00640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b3f0 tx=0x7f1b8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c015020 con 0x7f1b980b3b20 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c011830 con 0x7f1b980b3b20 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c011c20 con 0x7f1b980b3b20 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b98142820 con 0x7f1b980b3b20 2024-04-26T21:47:21.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.645+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b98142de0 con 0x7f1b980b3b20 2024-04-26T21:47:21.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.646+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b980693c0 con 0x7f1b980b3b20 2024-04-26T21:47:21.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.647+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8c018020 con 0x7f1b980b3b20 2024-04-26T21:47:21.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.648+0000 7f1b95600640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 0x7f1b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.648+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b8c08ba50 con 0x7f1b980b3b20 2024-04-26T21:47:21.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.648+0000 7f1b9d200640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 0x7f1b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.649+0000 7f1b9d200640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 0x7f1b7806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b8402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.653+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b8c01f020 con 0x7f1b980b3b20 2024-04-26T21:47:21.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.834+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b9815cdf0 con 0x7f1b7806c8f0 2024-04-26T21:47:21.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.853+0000 7f1b95600640 1 -- 172.21.15.106:0/4151196292 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f1b9815cdf0 con 0x7f1b7806c8f0 2024-04-26T21:47:21.856 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:21.857 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:47:21.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.857+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 msgr2=0x7f1b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.857+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 0x7f1b7806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b8402c000 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.857+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 msgr2=0x7f1b9815c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.857+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b3f0 tx=0x7f1b8c00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 shutdown_connections 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b98146680 0x7f1b9813fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b7806c8f0 0x7f1b7806edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b980d0e70 0x7f1b98143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 --2- 172.21.15.106:0/4151196292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b980b3b20 0x7f1b9815c510 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 >> 172.21.15.106:0/4151196292 conn(0x7f1b9806c740 msgr2=0x7f1b9815f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 shutdown_connections 2024-04-26T21:47:21.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:21.858+0000 7f1b9f000640 1 -- 172.21.15.106:0/4151196292 wait complete. 2024-04-26T21:47:21.874 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-26T21:47:21.113949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.669831"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302623+0000","last_change":"2024-04-26T21:47:15.501023+0000","last_active":"2024-04-26T21:47:18.302623+0000","last_peered":"2024-04-26T21:47:18.302623+0000","last_clean":"2024-04-26T21:47:18.302623+0000","last_became_active":"2024-04-26T21:47:15.500723+0000","last_became_peered":"2024-04-26T21:47:15.500723+0000","last_unstale":"2024-04-26T21:47:18.302623+0000","last_undegraded":"2024-04-26T21:47:18.302623+0000","last_fullsized":"2024-04-26T21:47:18.302623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:27.699043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300975+0000","last_change":"2024-04-26T21:47:15.506143+0000","last_active":"2024-04-26T21:47:18.300975+0000","last_peered":"2024-04-26T21:47:18.300975+0000","last_clean":"2024-04-26T21:47:18.300975+0000","last_became_active":"2024-04-26T21:47:15.501511+0000","last_became_peered":"2024-04-26T21:47:15.501511+0000","last_unstale":"2024-04-26T21:47:18.300975+0000","last_undegraded":"2024-04-26T21:47:18.300975+0000","last_fullsized":"2024-04-26T21:47:18.300975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:56.861793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302571+0000","last_change":"2024-04-26T21:47:15.507993+0000","last_active":"2024-04-26T21:47:18.302571+0000","last_peered":"2024-04-26T21:47:18.302571+0000","last_clean":"2024-04-26T21:47:18.302571+0000","last_became_active":"2024-04-26T21:47:15.502923+0000","last_became_peered":"2024-04-26T21:47:15.502923+0000","last_unstale":"2024-04-26T21:47:18.302571+0000","last_undegraded":"2024-04-26T21:47:18.302571+0000","last_fullsized":"2024-04-26T21:47:18.302571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495739+0000","last_change":"2024-04-26T21:47:15.499851+0000","last_active":"2024-04-26T21:47:16.495739+0000","last_peered":"2024-04-26T21:47:16.495739+0000","last_clean":"2024-04-26T21:47:16.495739+0000","last_became_active":"2024-04-26T21:47:15.498066+0000","last_became_peered":"2024-04-26T21:47:15.498066+0000","last_unstale":"2024-04-26T21:47:16.495739+0000","last_undegraded":"2024-04-26T21:47:16.495739+0000","last_fullsized":"2024-04-26T21:47:16.495739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.734287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595590+0000","last_change":"2024-04-26T21:47:15.500779+0000","last_active":"2024-04-26T21:47:16.595590+0000","last_peered":"2024-04-26T21:47:16.595590+0000","last_clean":"2024-04-26T21:47:16.595590+0000","last_became_active":"2024-04-26T21:47:15.500521+0000","last_became_peered":"2024-04-26T21:47:15.500521+0000","last_unstale":"2024-04-26T21:47:16.595590+0000","last_undegraded":"2024-04-26T21:47:16.595590+0000","last_fullsized":"2024-04-26T21:47:16.595590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:21.840377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299815+0000","last_change":"2024-04-26T21:47:15.488048+0000","last_active":"2024-04-26T21:47:18.299815+0000","last_peered":"2024-04-26T21:47:18.299815+0000","last_clean":"2024-04-26T21:47:18.299815+0000","last_became_active":"2024-04-26T21:47:15.487740+0000","last_became_peered":"2024-04-26T21:47:15.487740+0000","last_unstale":"2024-04-26T21:47:18.299815+0000","last_undegraded":"2024-04-26T21:47:18.299815+0000","last_fullsized":"2024-04-26T21:47:18.299815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497735+0000","last_change":"2024-04-26T21:47:15.500182+0000","last_active":"2024-04-26T21:47:16.497735+0000","last_peered":"2024-04-26T21:47:16.497735+0000","last_clean":"2024-04-26T21:47:16.497735+0000","last_became_active":"2024-04-26T21:47:15.499921+0000","last_became_peered":"2024-04-26T21:47:15.499921+0000","last_unstale":"2024-04-26T21:47:16.497735+0000","last_undegraded":"2024-04-26T21:47:16.497735+0000","last_fullsized":"2024-04-26T21:47:16.497735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.734287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498367+0000","last_change":"2024-04-26T21:47:15.499544+0000","last_active":"2024-04-26T21:47:16.498367+0000","last_peered":"2024-04-26T21:47:16.498367+0000","last_clean":"2024-04-26T21:47:16.498367+0000","last_became_active":"2024-04-26T21:47:15.497196+0000","last_became_peered":"2024-04-26T21:47:15.497196+0000","last_unstale":"2024-04-26T21:47:16.498367+0000","last_undegraded":"2024-04-26T21:47:16.498367+0000","last_fullsized":"2024-04-26T21:47:16.498367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302944+0000","last_change":"2024-04-26T21:47:15.503715+0000","last_active":"2024-04-26T21:47:18.302944+0000","last_peered":"2024-04-26T21:47:18.302944+0000","last_clean":"2024-04-26T21:47:18.302944+0000","last_became_active":"2024-04-26T21:47:15.503458+0000","last_became_peered":"2024-04-26T21:47:15.503458+0000","last_unstale":"2024-04-26T21:47:18.302944+0000","last_undegraded":"2024-04-26T21:47:18.302944+0000","last_fullsized":"2024-04-26T21:47:18.302944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:14.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496890+0000","last_change":"2024-04-26T21:47:15.501650+0000","last_active":"2024-04-26T21:47:16.496890+0000","last_peered":"2024-04-26T21:47:16.496890+0000","last_clean":"2024-04-26T21:47:16.496890+0000","last_became_active":"2024-04-26T21:47:15.501307+0000","last_became_peered":"2024-04-26T21:47:15.501307+0000","last_unstale":"2024-04-26T21:47:16.496890+0000","last_undegraded":"2024-04-26T21:47:16.496890+0000","last_fullsized":"2024-04-26T21:47:16.496890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498245+0000","last_change":"2024-04-26T21:47:15.505347+0000","last_active":"2024-04-26T21:47:16.498245+0000","last_peered":"2024-04-26T21:47:16.498245+0000","last_clean":"2024-04-26T21:47:16.498245+0000","last_became_active":"2024-04-26T21:47:15.505054+0000","last_became_peered":"2024-04-26T21:47:15.505054+0000","last_unstale":"2024-04-26T21:47:16.498245+0000","last_undegraded":"2024-04-26T21:47:16.498245+0000","last_fullsized":"2024-04-26T21:47:16.498245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303692+0000","last_change":"2024-04-26T21:47:15.496215+0000","last_active":"2024-04-26T21:47:18.303692+0000","last_peered":"2024-04-26T21:47:18.303692+0000","last_clean":"2024-04-26T21:47:18.303692+0000","last_became_active":"2024-04-26T21:47:15.494518+0000","last_became_peered":"2024-04-26T21:47:15.494518+0000","last_unstale":"2024-04-26T21:47:18.303692+0000","last_undegraded":"2024-04-26T21:47:18.303692+0000","last_fullsized":"2024-04-26T21:47:18.303692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595599+0000","last_change":"2024-04-26T21:47:15.506769+0000","last_active":"2024-04-26T21:47:16.595599+0000","last_peered":"2024-04-26T21:47:16.595599+0000","last_clean":"2024-04-26T21:47:16.595599+0000","last_became_active":"2024-04-26T21:47:15.506081+0000","last_became_peered":"2024-04-26T21:47:15.506081+0000","last_unstale":"2024-04-26T21:47:16.595599+0000","last_undegraded":"2024-04-26T21:47:16.595599+0000","last_fullsized":"2024-04-26T21:47:16.595599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:08.466342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300273+0000","last_change":"2024-04-26T21:47:15.499120+0000","last_active":"2024-04-26T21:47:18.300273+0000","last_peered":"2024-04-26T21:47:18.300273+0000","last_clean":"2024-04-26T21:47:18.300273+0000","last_became_active":"2024-04-26T21:47:15.497893+0000","last_became_peered":"2024-04-26T21:47:15.497893+0000","last_unstale":"2024-04-26T21:47:18.300273+0000","last_undegraded":"2024-04-26T21:47:18.300273+0000","last_fullsized":"2024-04-26T21:47:18.300273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300228+0000","last_change":"2024-04-26T21:47:15.510119+0000","last_active":"2024-04-26T21:47:18.300228+0000","last_peered":"2024-04-26T21:47:18.300228+0000","last_clean":"2024-04-26T21:47:18.300228+0000","last_became_active":"2024-04-26T21:47:15.509832+0000","last_became_peered":"2024-04-26T21:47:15.509832+0000","last_unstale":"2024-04-26T21:47:18.300228+0000","last_undegraded":"2024-04-26T21:47:18.300228+0000","last_fullsized":"2024-04-26T21:47:18.300228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498817+0000","last_change":"2024-04-26T21:47:15.488013+0000","last_active":"2024-04-26T21:47:16.498817+0000","last_peered":"2024-04-26T21:47:16.498817+0000","last_clean":"2024-04-26T21:47:16.498817+0000","last_became_active":"2024-04-26T21:47:15.487642+0000","last_became_peered":"2024-04-26T21:47:15.487642+0000","last_unstale":"2024-04-26T21:47:16.498817+0000","last_undegraded":"2024-04-26T21:47:16.498817+0000","last_fullsized":"2024-04-26T21:47:16.498817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498783+0000","last_change":"2024-04-26T21:47:15.505880+0000","last_active":"2024-04-26T21:47:16.498783+0000","last_peered":"2024-04-26T21:47:16.498783+0000","last_clean":"2024-04-26T21:47:16.498783+0000","last_became_active":"2024-04-26T21:47:15.505570+0000","last_became_peered":"2024-04-26T21:47:15.505570+0000","last_unstale":"2024-04-26T21:47:16.498783+0000","last_undegraded":"2024-04-26T21:47:16.498783+0000","last_fullsized":"2024-04-26T21:47:16.498783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300954+0000","last_change":"2024-04-26T21:47:15.505147+0000","last_active":"2024-04-26T21:47:18.300954+0000","last_peered":"2024-04-26T21:47:18.300954+0000","last_clean":"2024-04-26T21:47:18.300954+0000","last_became_active":"2024-04-26T21:47:15.504613+0000","last_became_peered":"2024-04-26T21:47:15.504613+0000","last_unstale":"2024-04-26T21:47:18.300954+0000","last_undegraded":"2024-04-26T21:47:18.300954+0000","last_fullsized":"2024-04-26T21:47:18.300954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:07.866270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300105+0000","last_change":"2024-04-26T21:47:15.504961+0000","last_active":"2024-04-26T21:47:18.300105+0000","last_peered":"2024-04-26T21:47:18.300105+0000","last_clean":"2024-04-26T21:47:18.300105+0000","last_became_active":"2024-04-26T21:47:15.504583+0000","last_became_peered":"2024-04-26T21:47:15.504583+0000","last_unstale":"2024-04-26T21:47:18.300105+0000","last_undegraded":"2024-04-26T21:47:18.300105+0000","last_fullsized":"2024-04-26T21:47:18.300105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595847+0000","last_change":"2024-04-26T21:47:15.498329+0000","last_active":"2024-04-26T21:47:16.595847+0000","last_peered":"2024-04-26T21:47:16.595847+0000","last_clean":"2024-04-26T21:47:16.595847+0000","last_became_active":"2024-04-26T21:47:15.497328+0000","last_became_peered":"2024-04-26T21:47:15.497328+0000","last_unstale":"2024-04-26T21:47:16.595847+0000","last_undegraded":"2024-04-26T21:47:16.595847+0000","last_fullsized":"2024-04-26T21:47:16.595847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:06.279320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301684+0000","last_change":"2024-04-26T21:47:15.510251+0000","last_active":"2024-04-26T21:47:18.301684+0000","last_peered":"2024-04-26T21:47:18.301684+0000","last_clean":"2024-04-26T21:47:18.301684+0000","last_became_active":"2024-04-26T21:47:15.508589+0000","last_became_peered":"2024-04-26T21:47:15.508589+0000","last_unstale":"2024-04-26T21:47:18.301684+0000","last_undegraded":"2024-04-26T21:47:18.301684+0000","last_fullsized":"2024-04-26T21:47:18.301684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.659159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303727+0000","last_change":"2024-04-26T21:47:15.493959+0000","last_active":"2024-04-26T21:47:18.303727+0000","last_peered":"2024-04-26T21:47:18.303727+0000","last_clean":"2024-04-26T21:47:18.303727+0000","last_became_active":"2024-04-26T21:47:15.493635+0000","last_became_peered":"2024-04-26T21:47:15.493635+0000","last_unstale":"2024-04-26T21:47:18.303727+0000","last_undegraded":"2024-04-26T21:47:18.303727+0000","last_fullsized":"2024-04-26T21:47:18.303727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:30.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498276+0000","last_change":"2024-04-26T21:47:15.501637+0000","last_active":"2024-04-26T21:47:16.498276+0000","last_peered":"2024-04-26T21:47:16.498276+0000","last_clean":"2024-04-26T21:47:16.498276+0000","last_became_active":"2024-04-26T21:47:15.501337+0000","last_became_peered":"2024-04-26T21:47:15.501337+0000","last_unstale":"2024-04-26T21:47:16.498276+0000","last_undegraded":"2024-04-26T21:47:16.498276+0000","last_fullsized":"2024-04-26T21:47:16.498276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:14.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299833+0000","last_change":"2024-04-26T21:47:15.498297+0000","last_active":"2024-04-26T21:47:18.299833+0000","last_peered":"2024-04-26T21:47:18.299833+0000","last_clean":"2024-04-26T21:47:18.299833+0000","last_became_active":"2024-04-26T21:47:15.497962+0000","last_became_peered":"2024-04-26T21:47:15.497962+0000","last_unstale":"2024-04-26T21:47:18.299833+0000","last_undegraded":"2024-04-26T21:47:18.299833+0000","last_fullsized":"2024-04-26T21:47:18.299833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496420+0000","last_change":"2024-04-26T21:47:15.503758+0000","last_active":"2024-04-26T21:47:16.496420+0000","last_peered":"2024-04-26T21:47:16.496420+0000","last_clean":"2024-04-26T21:47:16.496420+0000","last_became_active":"2024-04-26T21:47:15.503053+0000","last_became_peered":"2024-04-26T21:47:15.503053+0000","last_unstale":"2024-04-26T21:47:16.496420+0000","last_undegraded":"2024-04-26T21:47:16.496420+0000","last_fullsized":"2024-04-26T21:47:16.496420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301197+0000","last_change":"2024-04-26T21:47:15.488771+0000","last_active":"2024-04-26T21:47:18.301197+0000","last_peered":"2024-04-26T21:47:18.301197+0000","last_clean":"2024-04-26T21:47:18.301197+0000","last_became_active":"2024-04-26T21:47:15.488378+0000","last_became_peered":"2024-04-26T21:47:15.488378+0000","last_unstale":"2024-04-26T21:47:18.301197+0000","last_undegraded":"2024-04-26T21:47:18.301197+0000","last_fullsized":"2024-04-26T21:47:18.301197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:43.157703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496465+0000","last_change":"2024-04-26T21:47:15.512492+0000","last_active":"2024-04-26T21:47:16.496465+0000","last_peered":"2024-04-26T21:47:16.496465+0000","last_clean":"2024-04-26T21:47:16.496465+0000","last_became_active":"2024-04-26T21:47:15.511942+0000","last_became_peered":"2024-04-26T21:47:15.511942+0000","last_unstale":"2024-04-26T21:47:16.496465+0000","last_undegraded":"2024-04-26T21:47:16.496465+0000","last_fullsized":"2024-04-26T21:47:16.496465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303926+0000","last_change":"2024-04-26T21:47:15.500877+0000","last_active":"2024-04-26T21:47:18.303926+0000","last_peered":"2024-04-26T21:47:18.303926+0000","last_clean":"2024-04-26T21:47:18.303926+0000","last_became_active":"2024-04-26T21:47:15.500565+0000","last_became_peered":"2024-04-26T21:47:15.500565+0000","last_unstale":"2024-04-26T21:47:18.303926+0000","last_undegraded":"2024-04-26T21:47:18.303926+0000","last_fullsized":"2024-04-26T21:47:18.303926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302113+0000","last_change":"2024-04-26T21:47:15.492126+0000","last_active":"2024-04-26T21:47:18.302113+0000","last_peered":"2024-04-26T21:47:18.302113+0000","last_clean":"2024-04-26T21:47:18.302113+0000","last_became_active":"2024-04-26T21:47:15.491855+0000","last_became_peered":"2024-04-26T21:47:15.491855+0000","last_unstale":"2024-04-26T21:47:18.302113+0000","last_undegraded":"2024-04-26T21:47:18.302113+0000","last_fullsized":"2024-04-26T21:47:18.302113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:54.721735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497842+0000","last_change":"2024-04-26T21:47:15.499360+0000","last_active":"2024-04-26T21:47:16.497842+0000","last_peered":"2024-04-26T21:47:16.497842+0000","last_clean":"2024-04-26T21:47:16.497842+0000","last_became_active":"2024-04-26T21:47:15.499193+0000","last_became_peered":"2024-04-26T21:47:15.499193+0000","last_unstale":"2024-04-26T21:47:16.497842+0000","last_undegraded":"2024-04-26T21:47:16.497842+0000","last_fullsized":"2024-04-26T21:47:16.497842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301308+0000","last_change":"2024-04-26T21:47:15.507840+0000","last_active":"2024-04-26T21:47:18.301308+0000","last_peered":"2024-04-26T21:47:18.301308+0000","last_clean":"2024-04-26T21:47:18.301308+0000","last_became_active":"2024-04-26T21:47:15.507551+0000","last_became_peered":"2024-04-26T21:47:15.507551+0000","last_unstale":"2024-04-26T21:47:18.301308+0000","last_undegraded":"2024-04-26T21:47:18.301308+0000","last_fullsized":"2024-04-26T21:47:18.301308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:32.425606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497923+0000","last_change":"2024-04-26T21:47:15.483176+0000","last_active":"2024-04-26T21:47:16.497923+0000","last_peered":"2024-04-26T21:47:16.497923+0000","last_clean":"2024-04-26T21:47:16.497923+0000","last_became_active":"2024-04-26T21:47:15.481069+0000","last_became_peered":"2024-04-26T21:47:15.481069+0000","last_unstale":"2024-04-26T21:47:16.497923+0000","last_undegraded":"2024-04-26T21:47:16.497923+0000","last_fullsized":"2024-04-26T21:47:16.497923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498984+0000","last_change":"2024-04-26T21:47:15.499599+0000","last_active":"2024-04-26T21:47:16.498984+0000","last_peered":"2024-04-26T21:47:16.498984+0000","last_clean":"2024-04-26T21:47:16.498984+0000","last_became_active":"2024-04-26T21:47:15.499186+0000","last_became_peered":"2024-04-26T21:47:15.499186+0000","last_unstale":"2024-04-26T21:47:16.498984+0000","last_undegraded":"2024-04-26T21:47:16.498984+0000","last_fullsized":"2024-04-26T21:47:16.498984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:03.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303576+0000","last_change":"2024-04-26T21:47:14.483471+0000","last_active":"2024-04-26T21:47:18.303576+0000","last_peered":"2024-04-26T21:47:18.303576+0000","last_clean":"2024-04-26T21:47:18.303576+0000","last_became_active":"2024-04-26T21:47:14.483132+0000","last_became_peered":"2024-04-26T21:47:14.483132+0000","last_unstale":"2024-04-26T21:47:18.303576+0000","last_undegraded":"2024-04-26T21:47:18.303576+0000","last_fullsized":"2024-04-26T21:47:18.303576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303089+0000","last_change":"2024-04-26T21:47:15.499872+0000","last_active":"2024-04-26T21:47:18.303089+0000","last_peered":"2024-04-26T21:47:18.303089+0000","last_clean":"2024-04-26T21:47:18.303089+0000","last_became_active":"2024-04-26T21:47:15.499232+0000","last_became_peered":"2024-04-26T21:47:15.499232+0000","last_unstale":"2024-04-26T21:47:18.303089+0000","last_undegraded":"2024-04-26T21:47:18.303089+0000","last_fullsized":"2024-04-26T21:47:18.303089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:21.539767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499072+0000","last_change":"2024-04-26T21:47:14.477793+0000","last_active":"2024-04-26T21:47:16.499072+0000","last_peered":"2024-04-26T21:47:16.499072+0000","last_clean":"2024-04-26T21:47:16.499072+0000","last_became_active":"2024-04-26T21:47:14.477460+0000","last_became_peered":"2024-04-26T21:47:14.477460+0000","last_unstale":"2024-04-26T21:47:16.499072+0000","last_undegraded":"2024-04-26T21:47:16.499072+0000","last_fullsized":"2024-04-26T21:47:16.499072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303614+0000","last_change":"2024-04-26T21:47:15.495822+0000","last_active":"2024-04-26T21:47:18.303614+0000","last_peered":"2024-04-26T21:47:18.303614+0000","last_clean":"2024-04-26T21:47:18.303614+0000","last_became_active":"2024-04-26T21:47:15.495505+0000","last_became_peered":"2024-04-26T21:47:15.495505+0000","last_unstale":"2024-04-26T21:47:18.303614+0000","last_undegraded":"2024-04-26T21:47:18.303614+0000","last_fullsized":"2024-04-26T21:47:18.303614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:27.340597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300184+0000","last_change":"2024-04-26T21:47:14.484669+0000","last_active":"2024-04-26T21:47:18.300184+0000","last_peered":"2024-04-26T21:47:18.300184+0000","last_clean":"2024-04-26T21:47:18.300184+0000","last_became_active":"2024-04-26T21:47:14.483933+0000","last_became_peered":"2024-04-26T21:47:14.483933+0000","last_unstale":"2024-04-26T21:47:18.300184+0000","last_undegraded":"2024-04-26T21:47:18.300184+0000","last_fullsized":"2024-04-26T21:47:18.300184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496512+0000","last_change":"2024-04-26T21:47:15.512586+0000","last_active":"2024-04-26T21:47:16.496512+0000","last_peered":"2024-04-26T21:47:16.496512+0000","last_clean":"2024-04-26T21:47:16.496512+0000","last_became_active":"2024-04-26T21:47:15.512160+0000","last_became_peered":"2024-04-26T21:47:15.512160+0000","last_unstale":"2024-04-26T21:47:16.496512+0000","last_undegraded":"2024-04-26T21:47:16.496512+0000","last_fullsized":"2024-04-26T21:47:16.496512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595982+0000","last_change":"2024-04-26T21:47:14.462571+0000","last_active":"2024-04-26T21:47:16.595982+0000","last_peered":"2024-04-26T21:47:16.595982+0000","last_clean":"2024-04-26T21:47:16.595982+0000","last_became_active":"2024-04-26T21:47:14.461470+0000","last_became_peered":"2024-04-26T21:47:14.461470+0000","last_unstale":"2024-04-26T21:47:16.595982+0000","last_undegraded":"2024-04-26T21:47:16.595982+0000","last_fullsized":"2024-04-26T21:47:16.595982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:02.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498307+0000","last_change":"2024-04-26T21:47:15.480769+0000","last_active":"2024-04-26T21:47:16.498307+0000","last_peered":"2024-04-26T21:47:16.498307+0000","last_clean":"2024-04-26T21:47:16.498307+0000","last_became_active":"2024-04-26T21:47:15.480389+0000","last_became_peered":"2024-04-26T21:47:15.480389+0000","last_unstale":"2024-04-26T21:47:16.498307+0000","last_undegraded":"2024-04-26T21:47:16.498307+0000","last_fullsized":"2024-04-26T21:47:16.498307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302363+0000","last_change":"2024-04-26T21:47:14.485718+0000","last_active":"2024-04-26T21:47:18.302363+0000","last_peered":"2024-04-26T21:47:18.302363+0000","last_clean":"2024-04-26T21:47:18.302363+0000","last_became_active":"2024-04-26T21:47:14.485458+0000","last_became_peered":"2024-04-26T21:47:14.485458+0000","last_unstale":"2024-04-26T21:47:18.302363+0000","last_undegraded":"2024-04-26T21:47:18.302363+0000","last_fullsized":"2024-04-26T21:47:18.302363+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:29.947919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300532+0000","last_change":"2024-04-26T21:47:15.505533+0000","last_active":"2024-04-26T21:47:18.300532+0000","last_peered":"2024-04-26T21:47:18.300532+0000","last_clean":"2024-04-26T21:47:18.300532+0000","last_became_active":"2024-04-26T21:47:15.505213+0000","last_became_peered":"2024-04-26T21:47:15.505213+0000","last_unstale":"2024-04-26T21:47:18.300532+0000","last_undegraded":"2024-04-26T21:47:18.300532+0000","last_fullsized":"2024-04-26T21:47:18.300532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303990+0000","last_change":"2024-04-26T21:47:14.484537+0000","last_active":"2024-04-26T21:47:18.303990+0000","last_peered":"2024-04-26T21:47:18.303990+0000","last_clean":"2024-04-26T21:47:18.303990+0000","last_became_active":"2024-04-26T21:47:14.484218+0000","last_became_peered":"2024-04-26T21:47:14.484218+0000","last_unstale":"2024-04-26T21:47:18.303990+0000","last_undegraded":"2024-04-26T21:47:18.303990+0000","last_fullsized":"2024-04-26T21:47:18.303990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:09.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304148+0000","last_change":"2024-04-26T21:47:15.501225+0000","last_active":"2024-04-26T21:47:18.304148+0000","last_peered":"2024-04-26T21:47:18.304148+0000","last_clean":"2024-04-26T21:47:18.304148+0000","last_became_active":"2024-04-26T21:47:15.500826+0000","last_became_peered":"2024-04-26T21:47:15.500826+0000","last_unstale":"2024-04-26T21:47:18.304148+0000","last_undegraded":"2024-04-26T21:47:18.304148+0000","last_fullsized":"2024-04-26T21:47:18.304148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:03.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299589+0000","last_change":"2024-04-26T21:47:14.483496+0000","last_active":"2024-04-26T21:47:18.299589+0000","last_peered":"2024-04-26T21:47:18.299589+0000","last_clean":"2024-04-26T21:47:18.299589+0000","last_became_active":"2024-04-26T21:47:14.483201+0000","last_became_peered":"2024-04-26T21:47:14.483201+0000","last_unstale":"2024-04-26T21:47:18.299589+0000","last_undegraded":"2024-04-26T21:47:18.299589+0000","last_fullsized":"2024-04-26T21:47:18.299589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495421+0000","last_change":"2024-04-26T21:47:15.506056+0000","last_active":"2024-04-26T21:47:16.495421+0000","last_peered":"2024-04-26T21:47:16.495421+0000","last_clean":"2024-04-26T21:47:16.495421+0000","last_became_active":"2024-04-26T21:47:15.505645+0000","last_became_peered":"2024-04-26T21:47:15.505645+0000","last_unstale":"2024-04-26T21:47:16.495421+0000","last_undegraded":"2024-04-26T21:47:16.495421+0000","last_fullsized":"2024-04-26T21:47:16.495421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:17.697309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304021+0000","last_change":"2024-04-26T21:47:14.480135+0000","last_active":"2024-04-26T21:47:18.304021+0000","last_peered":"2024-04-26T21:47:18.304021+0000","last_clean":"2024-04-26T21:47:18.304021+0000","last_became_active":"2024-04-26T21:47:14.479762+0000","last_became_peered":"2024-04-26T21:47:14.479762+0000","last_unstale":"2024-04-26T21:47:18.304021+0000","last_undegraded":"2024-04-26T21:47:18.304021+0000","last_fullsized":"2024-04-26T21:47:18.304021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:00.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300813+0000","last_change":"2024-04-26T21:47:15.498265+0000","last_active":"2024-04-26T21:47:18.300813+0000","last_peered":"2024-04-26T21:47:18.300813+0000","last_clean":"2024-04-26T21:47:18.300813+0000","last_became_active":"2024-04-26T21:47:15.497885+0000","last_became_peered":"2024-04-26T21:47:15.497885+0000","last_unstale":"2024-04-26T21:47:18.300813+0000","last_undegraded":"2024-04-26T21:47:18.300813+0000","last_fullsized":"2024-04-26T21:47:18.300813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301177+0000","last_change":"2024-04-26T21:47:14.484137+0000","last_active":"2024-04-26T21:47:18.301177+0000","last_peered":"2024-04-26T21:47:18.301177+0000","last_clean":"2024-04-26T21:47:18.301177+0000","last_became_active":"2024-04-26T21:47:14.483547+0000","last_became_peered":"2024-04-26T21:47:14.483547+0000","last_unstale":"2024-04-26T21:47:18.301177+0000","last_undegraded":"2024-04-26T21:47:18.301177+0000","last_fullsized":"2024-04-26T21:47:18.301177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:26.685031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301124+0000","last_change":"2024-04-26T21:47:15.506733+0000","last_active":"2024-04-26T21:47:18.301124+0000","last_peered":"2024-04-26T21:47:18.301124+0000","last_clean":"2024-04-26T21:47:18.301124+0000","last_became_active":"2024-04-26T21:47:15.506287+0000","last_became_peered":"2024-04-26T21:47:15.506287+0000","last_unstale":"2024-04-26T21:47:18.301124+0000","last_undegraded":"2024-04-26T21:47:18.301124+0000","last_fullsized":"2024-04-26T21:47:18.301124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:09.857376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499211+0000","last_change":"2024-04-26T21:47:14.485542+0000","last_active":"2024-04-26T21:47:16.499211+0000","last_peered":"2024-04-26T21:47:16.499211+0000","last_clean":"2024-04-26T21:47:16.499211+0000","last_became_active":"2024-04-26T21:47:14.485231+0000","last_became_peered":"2024-04-26T21:47:14.485231+0000","last_unstale":"2024-04-26T21:47:16.499211+0000","last_undegraded":"2024-04-26T21:47:16.499211+0000","last_fullsized":"2024-04-26T21:47:16.499211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300552+0000","last_change":"2024-04-26T21:47:15.486843+0000","last_active":"2024-04-26T21:47:18.300552+0000","last_peered":"2024-04-26T21:47:18.300552+0000","last_clean":"2024-04-26T21:47:18.300552+0000","last_became_active":"2024-04-26T21:47:15.486560+0000","last_became_peered":"2024-04-26T21:47:15.486560+0000","last_unstale":"2024-04-26T21:47:18.300552+0000","last_undegraded":"2024-04-26T21:47:18.300552+0000","last_fullsized":"2024-04-26T21:47:18.300552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302825+0000","last_change":"2024-04-26T21:47:14.472755+0000","last_active":"2024-04-26T21:47:18.302825+0000","last_peered":"2024-04-26T21:47:18.302825+0000","last_clean":"2024-04-26T21:47:18.302825+0000","last_became_active":"2024-04-26T21:47:14.472547+0000","last_became_peered":"2024-04-26T21:47:14.472547+0000","last_unstale":"2024-04-26T21:47:18.302825+0000","last_undegraded":"2024-04-26T21:47:18.302825+0000","last_fullsized":"2024-04-26T21:47:18.302825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302172+0000","last_change":"2024-04-26T21:47:15.507039+0000","last_active":"2024-04-26T21:47:18.302172+0000","last_peered":"2024-04-26T21:47:18.302172+0000","last_clean":"2024-04-26T21:47:18.302172+0000","last_became_active":"2024-04-26T21:47:15.503211+0000","last_became_peered":"2024-04-26T21:47:15.503211+0000","last_unstale":"2024-04-26T21:47:18.302172+0000","last_undegraded":"2024-04-26T21:47:18.302172+0000","last_fullsized":"2024-04-26T21:47:18.302172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300534+0000","last_change":"2024-04-26T21:47:14.487095+0000","last_active":"2024-04-26T21:47:18.300534+0000","last_peered":"2024-04-26T21:47:18.300534+0000","last_clean":"2024-04-26T21:47:18.300534+0000","last_became_active":"2024-04-26T21:47:14.486802+0000","last_became_peered":"2024-04-26T21:47:14.486802+0000","last_unstale":"2024-04-26T21:47:18.300534+0000","last_undegraded":"2024-04-26T21:47:18.300534+0000","last_fullsized":"2024-04-26T21:47:18.300534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.645147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495499+0000","last_change":"2024-04-26T21:47:15.505778+0000","last_active":"2024-04-26T21:47:16.495499+0000","last_peered":"2024-04-26T21:47:16.495499+0000","last_clean":"2024-04-26T21:47:16.495499+0000","last_became_active":"2024-04-26T21:47:15.505505+0000","last_became_peered":"2024-04-26T21:47:15.505505+0000","last_unstale":"2024-04-26T21:47:16.495499+0000","last_undegraded":"2024-04-26T21:47:16.495499+0000","last_fullsized":"2024-04-26T21:47:16.495499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:13.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302680+0000","last_change":"2024-04-26T21:47:14.478529+0000","last_active":"2024-04-26T21:47:18.302680+0000","last_peered":"2024-04-26T21:47:18.302680+0000","last_clean":"2024-04-26T21:47:18.302680+0000","last_became_active":"2024-04-26T21:47:14.478281+0000","last_became_peered":"2024-04-26T21:47:14.478281+0000","last_unstale":"2024-04-26T21:47:18.302680+0000","last_undegraded":"2024-04-26T21:47:18.302680+0000","last_fullsized":"2024-04-26T21:47:18.302680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300428+0000","last_change":"2024-04-26T21:47:15.506746+0000","last_active":"2024-04-26T21:47:18.300428+0000","last_peered":"2024-04-26T21:47:18.300428+0000","last_clean":"2024-04-26T21:47:18.300428+0000","last_became_active":"2024-04-26T21:47:15.506474+0000","last_became_peered":"2024-04-26T21:47:15.506474+0000","last_unstale":"2024-04-26T21:47:18.300428+0000","last_undegraded":"2024-04-26T21:47:18.300428+0000","last_fullsized":"2024-04-26T21:47:18.300428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495576+0000","last_change":"2024-04-26T21:47:14.480124+0000","last_active":"2024-04-26T21:47:16.495576+0000","last_peered":"2024-04-26T21:47:16.495576+0000","last_clean":"2024-04-26T21:47:16.495576+0000","last_became_active":"2024-04-26T21:47:14.479846+0000","last_became_peered":"2024-04-26T21:47:14.479846+0000","last_unstale":"2024-04-26T21:47:16.495576+0000","last_undegraded":"2024-04-26T21:47:16.495576+0000","last_fullsized":"2024-04-26T21:47:16.495576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498481+0000","last_change":"2024-04-26T21:47:15.482410+0000","last_active":"2024-04-26T21:47:16.498481+0000","last_peered":"2024-04-26T21:47:16.498481+0000","last_clean":"2024-04-26T21:47:16.498481+0000","last_became_active":"2024-04-26T21:47:15.480603+0000","last_became_peered":"2024-04-26T21:47:15.480603+0000","last_unstale":"2024-04-26T21:47:16.498481+0000","last_undegraded":"2024-04-26T21:47:16.498481+0000","last_fullsized":"2024-04-26T21:47:16.498481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299325+0000","last_change":"2024-04-26T21:47:14.485004+0000","last_active":"2024-04-26T21:47:18.299325+0000","last_peered":"2024-04-26T21:47:18.299325+0000","last_clean":"2024-04-26T21:47:18.299325+0000","last_became_active":"2024-04-26T21:47:14.483355+0000","last_became_peered":"2024-04-26T21:47:14.483355+0000","last_unstale":"2024-04-26T21:47:18.299325+0000","last_undegraded":"2024-04-26T21:47:18.299325+0000","last_fullsized":"2024-04-26T21:47:18.299325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301721+0000","last_change":"2024-04-26T21:47:15.511291+0000","last_active":"2024-04-26T21:47:18.301721+0000","last_peered":"2024-04-26T21:47:18.301721+0000","last_clean":"2024-04-26T21:47:18.301721+0000","last_became_active":"2024-04-26T21:47:15.509717+0000","last_became_peered":"2024-04-26T21:47:15.509717+0000","last_unstale":"2024-04-26T21:47:18.301721+0000","last_undegraded":"2024-04-26T21:47:18.301721+0000","last_fullsized":"2024-04-26T21:47:18.301721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.659159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299751+0000","last_change":"2024-04-26T21:47:14.482285+0000","last_active":"2024-04-26T21:47:18.299751+0000","last_peered":"2024-04-26T21:47:18.299751+0000","last_clean":"2024-04-26T21:47:18.299751+0000","last_became_active":"2024-04-26T21:47:14.481952+0000","last_became_peered":"2024-04-26T21:47:14.481952+0000","last_unstale":"2024-04-26T21:47:18.299751+0000","last_undegraded":"2024-04-26T21:47:18.299751+0000","last_fullsized":"2024-04-26T21:47:18.299751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:53.707723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301622+0000","last_change":"2024-04-26T21:47:15.486505+0000","last_active":"2024-04-26T21:47:18.301622+0000","last_peered":"2024-04-26T21:47:18.301622+0000","last_clean":"2024-04-26T21:47:18.301622+0000","last_became_active":"2024-04-26T21:47:15.486070+0000","last_became_peered":"2024-04-26T21:47:15.486070+0000","last_unstale":"2024-04-26T21:47:18.301622+0000","last_undegraded":"2024-04-26T21:47:18.301622+0000","last_fullsized":"2024-04-26T21:47:18.301622+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:21.539767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496639+0000","last_change":"2024-04-26T21:47:14.485819+0000","last_active":"2024-04-26T21:47:16.496639+0000","last_peered":"2024-04-26T21:47:16.496639+0000","last_clean":"2024-04-26T21:47:16.496639+0000","last_became_active":"2024-04-26T21:47:14.485435+0000","last_became_peered":"2024-04-26T21:47:14.485435+0000","last_unstale":"2024-04-26T21:47:16.496639+0000","last_undegraded":"2024-04-26T21:47:16.496639+0000","last_fullsized":"2024-04-26T21:47:16.496639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301477+0000","last_change":"2024-04-26T21:47:15.489120+0000","last_active":"2024-04-26T21:47:18.301477+0000","last_peered":"2024-04-26T21:47:18.301477+0000","last_clean":"2024-04-26T21:47:18.301477+0000","last_became_active":"2024-04-26T21:47:15.488799+0000","last_became_peered":"2024-04-26T21:47:15.488799+0000","last_unstale":"2024-04-26T21:47:18.301477+0000","last_undegraded":"2024-04-26T21:47:18.301477+0000","last_fullsized":"2024-04-26T21:47:18.301477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301934+0000","last_change":"2024-04-26T21:47:14.481662+0000","last_active":"2024-04-26T21:47:18.301934+0000","last_peered":"2024-04-26T21:47:18.301934+0000","last_clean":"2024-04-26T21:47:18.301934+0000","last_became_active":"2024-04-26T21:47:14.480582+0000","last_became_peered":"2024-04-26T21:47:14.480582+0000","last_unstale":"2024-04-26T21:47:18.301934+0000","last_undegraded":"2024-04-26T21:47:18.301934+0000","last_fullsized":"2024-04-26T21:47:18.301934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496000+0000","last_change":"2024-04-26T21:47:15.499588+0000","last_active":"2024-04-26T21:47:16.496000+0000","last_peered":"2024-04-26T21:47:16.496000+0000","last_clean":"2024-04-26T21:47:16.496000+0000","last_became_active":"2024-04-26T21:47:15.496595+0000","last_became_peered":"2024-04-26T21:47:15.496595+0000","last_unstale":"2024-04-26T21:47:16.496000+0000","last_undegraded":"2024-04-26T21:47:16.496000+0000","last_fullsized":"2024-04-26T21:47:16.496000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304442+0000","last_change":"2024-04-26T21:47:14.475423+0000","last_active":"2024-04-26T21:47:18.304442+0000","last_peered":"2024-04-26T21:47:18.304442+0000","last_clean":"2024-04-26T21:47:18.304442+0000","last_became_active":"2024-04-26T21:47:14.475083+0000","last_became_peered":"2024-04-26T21:47:14.475083+0000","last_unstale":"2024-04-26T21:47:18.304442+0000","last_undegraded":"2024-04-26T21:47:18.304442+0000","last_fullsized":"2024-04-26T21:47:18.304442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:20.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.596005+0000","last_change":"2024-04-26T21:47:15.499100+0000","last_active":"2024-04-26T21:47:16.596005+0000","last_peered":"2024-04-26T21:47:16.596005+0000","last_clean":"2024-04-26T21:47:16.596005+0000","last_became_active":"2024-04-26T21:47:15.498023+0000","last_became_peered":"2024-04-26T21:47:15.498023+0000","last_unstale":"2024-04-26T21:47:16.596005+0000","last_undegraded":"2024-04-26T21:47:16.596005+0000","last_fullsized":"2024-04-26T21:47:16.596005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:55.763012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303875+0000","last_change":"2024-04-26T21:47:14.487648+0000","last_active":"2024-04-26T21:47:18.303875+0000","last_peered":"2024-04-26T21:47:18.303875+0000","last_clean":"2024-04-26T21:47:18.303875+0000","last_became_active":"2024-04-26T21:47:14.487356+0000","last_became_peered":"2024-04-26T21:47:14.487356+0000","last_unstale":"2024-04-26T21:47:18.303875+0000","last_undegraded":"2024-04-26T21:47:18.303875+0000","last_fullsized":"2024-04-26T21:47:18.303875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:15.031293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595646+0000","last_change":"2024-04-26T21:47:15.503358+0000","last_active":"2024-04-26T21:47:16.595646+0000","last_peered":"2024-04-26T21:47:16.595646+0000","last_clean":"2024-04-26T21:47:16.595646+0000","last_became_active":"2024-04-26T21:47:15.502980+0000","last_became_peered":"2024-04-26T21:47:15.502980+0000","last_unstale":"2024-04-26T21:47:16.595646+0000","last_undegraded":"2024-04-26T21:47:16.595646+0000","last_fullsized":"2024-04-26T21:47:16.595646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:30.961931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498624+0000","last_change":"2024-04-26T21:47:14.478983+0000","last_active":"2024-04-26T21:47:16.498624+0000","last_peered":"2024-04-26T21:47:16.498624+0000","last_clean":"2024-04-26T21:47:16.498624+0000","last_became_active":"2024-04-26T21:47:14.478698+0000","last_became_peered":"2024-04-26T21:47:14.478698+0000","last_unstale":"2024-04-26T21:47:16.498624+0000","last_undegraded":"2024-04-26T21:47:16.498624+0000","last_fullsized":"2024-04-26T21:47:16.498624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595732+0000","last_change":"2024-04-26T21:47:15.507004+0000","last_active":"2024-04-26T21:47:16.595732+0000","last_peered":"2024-04-26T21:47:16.595732+0000","last_clean":"2024-04-26T21:47:16.595732+0000","last_became_active":"2024-04-26T21:47:15.506486+0000","last_became_peered":"2024-04-26T21:47:15.506486+0000","last_unstale":"2024-04-26T21:47:16.595732+0000","last_undegraded":"2024-04-26T21:47:16.595732+0000","last_fullsized":"2024-04-26T21:47:16.595732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:21.290574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302825+0000","last_change":"2024-04-26T21:47:14.482597+0000","last_active":"2024-04-26T21:47:18.302825+0000","last_peered":"2024-04-26T21:47:18.302825+0000","last_clean":"2024-04-26T21:47:18.302825+0000","last_became_active":"2024-04-26T21:47:14.482299+0000","last_became_peered":"2024-04-26T21:47:14.482299+0000","last_unstale":"2024-04-26T21:47:18.302825+0000","last_undegraded":"2024-04-26T21:47:18.302825+0000","last_fullsized":"2024-04-26T21:47:18.302825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595420+0000","last_change":"2024-04-26T21:47:15.505684+0000","last_active":"2024-04-26T21:47:16.595420+0000","last_peered":"2024-04-26T21:47:16.595420+0000","last_clean":"2024-04-26T21:47:16.595420+0000","last_became_active":"2024-04-26T21:47:15.505416+0000","last_became_peered":"2024-04-26T21:47:15.505416+0000","last_unstale":"2024-04-26T21:47:16.595420+0000","last_undegraded":"2024-04-26T21:47:16.595420+0000","last_fullsized":"2024-04-26T21:47:16.595420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:10.577425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301724+0000","last_change":"2024-04-26T21:47:14.475056+0000","last_active":"2024-04-26T21:47:18.301724+0000","last_peered":"2024-04-26T21:47:18.301724+0000","last_clean":"2024-04-26T21:47:18.301724+0000","last_became_active":"2024-04-26T21:47:14.474735+0000","last_became_peered":"2024-04-26T21:47:14.474735+0000","last_unstale":"2024-04-26T21:47:18.301724+0000","last_undegraded":"2024-04-26T21:47:18.301724+0000","last_fullsized":"2024-04-26T21:47:18.301724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:58.661753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497558+0000","last_change":"2024-04-26T21:47:15.509371+0000","last_active":"2024-04-26T21:47:16.497558+0000","last_peered":"2024-04-26T21:47:16.497558+0000","last_clean":"2024-04-26T21:47:16.497558+0000","last_became_active":"2024-04-26T21:47:15.508964+0000","last_became_peered":"2024-04-26T21:47:15.508964+0000","last_unstale":"2024-04-26T21:47:16.497558+0000","last_undegraded":"2024-04-26T21:47:16.497558+0000","last_fullsized":"2024-04-26T21:47:16.497558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:13.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299714+0000","last_change":"2024-04-26T21:47:14.478761+0000","last_active":"2024-04-26T21:47:18.299714+0000","last_peered":"2024-04-26T21:47:18.299714+0000","last_clean":"2024-04-26T21:47:18.299714+0000","last_became_active":"2024-04-26T21:47:14.477387+0000","last_became_peered":"2024-04-26T21:47:14.477387+0000","last_unstale":"2024-04-26T21:47:18.299714+0000","last_undegraded":"2024-04-26T21:47:18.299714+0000","last_fullsized":"2024-04-26T21:47:18.299714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595837+0000","last_change":"2024-04-26T21:47:15.497721+0000","last_active":"2024-04-26T21:47:16.595837+0000","last_peered":"2024-04-26T21:47:16.595837+0000","last_clean":"2024-04-26T21:47:16.595837+0000","last_became_active":"2024-04-26T21:47:15.497389+0000","last_became_peered":"2024-04-26T21:47:15.497389+0000","last_unstale":"2024-04-26T21:47:16.595837+0000","last_undegraded":"2024-04-26T21:47:16.595837+0000","last_fullsized":"2024-04-26T21:47:16.595837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:42.246393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496744+0000","last_change":"2024-04-26T21:47:14.476244+0000","last_active":"2024-04-26T21:47:16.496744+0000","last_peered":"2024-04-26T21:47:16.496744+0000","last_clean":"2024-04-26T21:47:16.496744+0000","last_became_active":"2024-04-26T21:47:14.475929+0000","last_became_peered":"2024-04-26T21:47:14.475929+0000","last_unstale":"2024-04-26T21:47:16.496744+0000","last_undegraded":"2024-04-26T21:47:16.496744+0000","last_fullsized":"2024-04-26T21:47:16.496744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301935+0000","last_change":"2024-04-26T21:47:15.505586+0000","last_active":"2024-04-26T21:47:18.301935+0000","last_peered":"2024-04-26T21:47:18.301935+0000","last_clean":"2024-04-26T21:47:18.301935+0000","last_became_active":"2024-04-26T21:47:15.505306+0000","last_became_peered":"2024-04-26T21:47:15.505306+0000","last_unstale":"2024-04-26T21:47:18.301935+0000","last_undegraded":"2024-04-26T21:47:18.301935+0000","last_fullsized":"2024-04-26T21:47:18.301935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499190+0000","last_change":"2024-04-26T21:47:14.479190+0000","last_active":"2024-04-26T21:47:16.499190+0000","last_peered":"2024-04-26T21:47:16.499190+0000","last_clean":"2024-04-26T21:47:16.499190+0000","last_became_active":"2024-04-26T21:47:14.478861+0000","last_became_peered":"2024-04-26T21:47:14.478861+0000","last_unstale":"2024-04-26T21:47:16.499190+0000","last_undegraded":"2024-04-26T21:47:16.499190+0000","last_fullsized":"2024-04-26T21:47:16.499190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498740+0000","last_change":"2024-04-26T21:47:15.507270+0000","last_active":"2024-04-26T21:47:16.498740+0000","last_peered":"2024-04-26T21:47:16.498740+0000","last_clean":"2024-04-26T21:47:16.498740+0000","last_became_active":"2024-04-26T21:47:15.505462+0000","last_became_peered":"2024-04-26T21:47:15.505462+0000","last_unstale":"2024-04-26T21:47:16.498740+0000","last_undegraded":"2024-04-26T21:47:16.498740+0000","last_fullsized":"2024-04-26T21:47:16.498740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:27.340597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496078+0000","last_change":"2024-04-26T21:47:14.465459+0000","last_active":"2024-04-26T21:47:16.496078+0000","last_peered":"2024-04-26T21:47:16.496078+0000","last_clean":"2024-04-26T21:47:16.496078+0000","last_became_active":"2024-04-26T21:47:14.464667+0000","last_became_peered":"2024-04-26T21:47:14.464667+0000","last_unstale":"2024-04-26T21:47:16.496078+0000","last_undegraded":"2024-04-26T21:47:16.496078+0000","last_fullsized":"2024-04-26T21:47:16.496078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302583+0000","last_change":"2024-04-26T21:47:15.506707+0000","last_active":"2024-04-26T21:47:18.302583+0000","last_peered":"2024-04-26T21:47:18.302583+0000","last_clean":"2024-04-26T21:47:18.302583+0000","last_became_active":"2024-04-26T21:47:15.506405+0000","last_became_peered":"2024-04-26T21:47:15.506405+0000","last_unstale":"2024-04-26T21:47:18.302583+0000","last_undegraded":"2024-04-26T21:47:18.302583+0000","last_fullsized":"2024-04-26T21:47:18.302583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.906497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498915+0000","last_change":"2024-04-26T21:47:14.481643+0000","last_active":"2024-04-26T21:47:16.498915+0000","last_peered":"2024-04-26T21:47:16.498915+0000","last_clean":"2024-04-26T21:47:16.498915+0000","last_became_active":"2024-04-26T21:47:14.481119+0000","last_became_peered":"2024-04-26T21:47:14.481119+0000","last_unstale":"2024-04-26T21:47:16.498915+0000","last_undegraded":"2024-04-26T21:47:16.498915+0000","last_fullsized":"2024-04-26T21:47:16.498915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498754+0000","last_change":"2024-04-26T21:47:15.511552+0000","last_active":"2024-04-26T21:47:16.498754+0000","last_peered":"2024-04-26T21:47:16.498754+0000","last_clean":"2024-04-26T21:47:16.498754+0000","last_became_active":"2024-04-26T21:47:15.511178+0000","last_became_peered":"2024-04-26T21:47:15.511178+0000","last_unstale":"2024-04-26T21:47:16.498754+0000","last_undegraded":"2024-04-26T21:47:16.498754+0000","last_fullsized":"2024-04-26T21:47:16.498754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:30.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495610+0000","last_change":"2024-04-26T21:47:14.475520+0000","last_active":"2024-04-26T21:47:16.495610+0000","last_peered":"2024-04-26T21:47:16.495610+0000","last_clean":"2024-04-26T21:47:16.495610+0000","last_became_active":"2024-04-26T21:47:14.475208+0000","last_became_peered":"2024-04-26T21:47:14.475208+0000","last_unstale":"2024-04-26T21:47:16.495610+0000","last_undegraded":"2024-04-26T21:47:16.495610+0000","last_fullsized":"2024-04-26T21:47:16.495610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299460+0000","last_change":"2024-04-26T21:47:15.493968+0000","last_active":"2024-04-26T21:47:18.299460+0000","last_peered":"2024-04-26T21:47:18.299460+0000","last_clean":"2024-04-26T21:47:18.299460+0000","last_became_active":"2024-04-26T21:47:15.493681+0000","last_became_peered":"2024-04-26T21:47:15.493681+0000","last_unstale":"2024-04-26T21:47:18.299460+0000","last_undegraded":"2024-04-26T21:47:18.299460+0000","last_fullsized":"2024-04-26T21:47:18.299460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301888+0000","last_change":"2024-04-26T21:47:14.477761+0000","last_active":"2024-04-26T21:47:18.301888+0000","last_peered":"2024-04-26T21:47:18.301888+0000","last_clean":"2024-04-26T21:47:18.301888+0000","last_became_active":"2024-04-26T21:47:14.477292+0000","last_became_peered":"2024-04-26T21:47:14.477292+0000","last_unstale":"2024-04-26T21:47:18.301888+0000","last_undegraded":"2024-04-26T21:47:18.301888+0000","last_fullsized":"2024-04-26T21:47:18.301888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:20.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301469+0000","last_change":"2024-04-26T21:47:15.494026+0000","last_active":"2024-04-26T21:47:18.301469+0000","last_peered":"2024-04-26T21:47:18.301469+0000","last_clean":"2024-04-26T21:47:18.301469+0000","last_became_active":"2024-04-26T21:47:15.493682+0000","last_became_peered":"2024-04-26T21:47:15.493682+0000","last_unstale":"2024-04-26T21:47:18.301469+0000","last_undegraded":"2024-04-26T21:47:18.301469+0000","last_fullsized":"2024-04-26T21:47:18.301469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:27.699043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299529+0000","last_change":"2024-04-26T21:47:14.485177+0000","last_active":"2024-04-26T21:47:18.299529+0000","last_peered":"2024-04-26T21:47:18.299529+0000","last_clean":"2024-04-26T21:47:18.299529+0000","last_became_active":"2024-04-26T21:47:14.483874+0000","last_became_peered":"2024-04-26T21:47:14.483874+0000","last_unstale":"2024-04-26T21:47:18.299529+0000","last_undegraded":"2024-04-26T21:47:18.299529+0000","last_fullsized":"2024-04-26T21:47:18.299529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497850+0000","last_change":"2024-04-26T21:47:15.497283+0000","last_active":"2024-04-26T21:47:16.497850+0000","last_peered":"2024-04-26T21:47:16.497850+0000","last_clean":"2024-04-26T21:47:16.497850+0000","last_became_active":"2024-04-26T21:47:15.497045+0000","last_became_peered":"2024-04-26T21:47:15.497045+0000","last_unstale":"2024-04-26T21:47:16.497850+0000","last_undegraded":"2024-04-26T21:47:16.497850+0000","last_fullsized":"2024-04-26T21:47:16.497850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301713+0000","last_change":"2024-04-26T21:47:14.480541+0000","last_active":"2024-04-26T21:47:18.301713+0000","last_peered":"2024-04-26T21:47:18.301713+0000","last_clean":"2024-04-26T21:47:18.301713+0000","last_became_active":"2024-04-26T21:47:14.480243+0000","last_became_peered":"2024-04-26T21:47:14.480243+0000","last_unstale":"2024-04-26T21:47:18.301713+0000","last_undegraded":"2024-04-26T21:47:18.301713+0000","last_fullsized":"2024-04-26T21:47:18.301713+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498753+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:16.498753+0000","last_peered":"2024-04-26T21:47:16.498753+0000","last_clean":"2024-04-26T21:47:16.498753+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:16.498753+0000","last_undegraded":"2024-04-26T21:47:16.498753+0000","last_fullsized":"2024-04-26T21:47:16.498753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:24.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:25.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.090+0000 7fbd26c00640 1 Processor -- start 2024-04-26T21:47:25.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.090+0000 7fbd26c00640 1 -- start start 2024-04-26T21:47:25.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.091+0000 7fbd26c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd200d5990 0x7fbd20163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.091+0000 7fbd26c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd200b82b0 0x7fbd200b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd200d3f50 con 0x7fbd2015f1b0 2024-04-26T21:47:25.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbd200d36b0 con 0x7fbd200b82b0 2024-04-26T21:47:25.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbd200d3990 con 0x7fbd200d5990 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd200b82b0 0x7fbd200b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd200b82b0 0x7fbd200b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53576/0 (socket says 172.21.15.106:53576) 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd26200640 1 -- 172.21.15.106:0/3622760737 learned_addr learned my addr 172.21.15.106:0/3622760737 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.092+0000 7fbd24e00640 1 --2- 172.21.15.106:0/3622760737 >> v2:172.21.15.204:3300/0 conn(0x7fbd200d5990 0x7fbd20163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.093+0000 7fbd26200640 1 -- 172.21.15.106:0/3622760737 >> v2:172.21.15.204:3300/0 conn(0x7fbd200d5990 msgr2=0x7fbd20163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.093+0000 7fbd26200640 1 --2- 172.21.15.106:0/3622760737 >> v2:172.21.15.204:3300/0 conn(0x7fbd200d5990 0x7fbd20163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.093+0000 7fbd26200640 1 -- 172.21.15.106:0/3622760737 >> v1:172.21.15.204:6789/0 conn(0x7fbd2015f1b0 legacy=0x7fbd20162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.093+0000 7fbd26200640 1 -- 172.21.15.106:0/3622760737 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd200d3c70 con 0x7fbd200b82b0 2024-04-26T21:47:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.094+0000 7fbd26200640 1 --2- 172.21.15.106:0/3622760737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd200b8740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009e50 tx=0x7fbd1802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38f443e376bb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.094+0000 7fbd1fe00640 1 -- 172.21.15.106:0/3622760737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18036070 con 0x7fbd200b82b0 2024-04-26T21:47:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd1fe00640 1 -- 172.21.15.106:0/3622760737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd18031600 con 0x7fbd200b82b0 2024-04-26T21:47:25.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd1fe00640 1 -- 172.21.15.106:0/3622760737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180319d0 con 0x7fbd200b82b0 2024-04-26T21:47:25.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 -- 172.21.15.106:0/3622760737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 msgr2=0x7fbd200b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3622760737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd200b8740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009e50 tx=0x7fbd1802a760 comp rx=0 tx=0).stop 2024-04-26T21:47:25.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 -- 172.21.15.106:0/3622760737 shutdown_connections 2024-04-26T21:47:25.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3622760737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd200b8740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3622760737 >> v2:172.21.15.204:3300/0 conn(0x7fbd200d5990 0x7fbd20163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 -- 172.21.15.106:0/3622760737 >> 172.21.15.106:0/3622760737 conn(0x7fbd2009e230 msgr2=0x7fbd200d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 -- 172.21.15.106:0/3622760737 shutdown_connections 2024-04-26T21:47:25.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.095+0000 7fbd26c00640 1 -- 172.21.15.106:0/3622760737 wait complete. 2024-04-26T21:47:25.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.097+0000 7fbd26c00640 1 Processor -- start 2024-04-26T21:47:25.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.097+0000 7fbd26c00640 1 -- start start 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 0x7fbd20146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd2015f1b0 0x7fbd201470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20163750 con 0x7fbd200b82b0 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd200d33a0 con 0x7fbd200d5990 2024-04-26T21:47:25.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd26c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd201638d0 con 0x7fbd2015f1b0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd25800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd25800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53592/0 (socket says 172.21.15.106:53592) 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.098+0000 7fbd25800640 1 -- 172.21.15.106:0/3576109130 learned_addr learned my addr 172.21.15.106:0/3576109130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd24e00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 0x7fbd20146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd26200640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd2015f1b0 0x7fbd201470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 -- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd2015f1b0 msgr2=0x7fbd201470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd2015f1b0 0x7fbd201470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 -- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 msgr2=0x7fbd20146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 0x7fbd20146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 -- 172.21.15.106:0/3576109130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201611e0 con 0x7fbd200b82b0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd24e00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 0x7fbd20146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.099+0000 7fbd25800640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c3f0 tx=0x7fbd1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.100+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd140052d0 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.100+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd14005490 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.100+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14007260 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.100+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd200cb640 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.100+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd20149910 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.101+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd200693c0 con 0x7fbd200b82b0 2024-04-26T21:47:25.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.102+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd14007400 con 0x7fbd200b82b0 2024-04-26T21:47:25.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.103+0000 7fbd1d600640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.103+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbd1408aff0 con 0x7fbd200b82b0 2024-04-26T21:47:25.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.103+0000 7fbd24e00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.104+0000 7fbd24e00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd2015cea0 tx=0x7fbd0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.107+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd140593c0 con 0x7fbd200b82b0 2024-04-26T21:47:25.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.309+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd2008d090 con 0x7fbd0006c8f0 2024-04-26T21:47:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.316+0000 7fbd1d600640 1 -- 172.21.15.106:0/3576109130 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7fbd2008d090 con 0x7fbd0006c8f0 2024-04-26T21:47:25.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:25.321 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:47:25.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 msgr2=0x7fbd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd2015cea0 tx=0x7fbd0c008040 comp rx=0 tx=0).stop 2024-04-26T21:47:25.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 msgr2=0x7fbd20149070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c3f0 tx=0x7fbd1400c8b0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 shutdown_connections 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd2015f1b0 0x7fbd201470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200d5990 0x7fbd20146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 --2- 172.21.15.106:0/3576109130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd200b82b0 0x7fbd20149070 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.321+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 >> 172.21.15.106:0/3576109130 conn(0x7fbd2009e230 msgr2=0x7fbd200d4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.322+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 shutdown_connections 2024-04-26T21:47:25.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.322+0000 7fbd26c00640 1 -- 172.21.15.106:0/3576109130 wait complete. 2024-04-26T21:47:25.340 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T21:47:25.115656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.647470"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302623+0000","last_change":"2024-04-26T21:47:15.501023+0000","last_active":"2024-04-26T21:47:18.302623+0000","last_peered":"2024-04-26T21:47:18.302623+0000","last_clean":"2024-04-26T21:47:18.302623+0000","last_became_active":"2024-04-26T21:47:15.500723+0000","last_became_peered":"2024-04-26T21:47:15.500723+0000","last_unstale":"2024-04-26T21:47:18.302623+0000","last_undegraded":"2024-04-26T21:47:18.302623+0000","last_fullsized":"2024-04-26T21:47:18.302623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:27.699043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300975+0000","last_change":"2024-04-26T21:47:15.506143+0000","last_active":"2024-04-26T21:47:18.300975+0000","last_peered":"2024-04-26T21:47:18.300975+0000","last_clean":"2024-04-26T21:47:18.300975+0000","last_became_active":"2024-04-26T21:47:15.501511+0000","last_became_peered":"2024-04-26T21:47:15.501511+0000","last_unstale":"2024-04-26T21:47:18.300975+0000","last_undegraded":"2024-04-26T21:47:18.300975+0000","last_fullsized":"2024-04-26T21:47:18.300975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:56.861793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302571+0000","last_change":"2024-04-26T21:47:15.507993+0000","last_active":"2024-04-26T21:47:18.302571+0000","last_peered":"2024-04-26T21:47:18.302571+0000","last_clean":"2024-04-26T21:47:18.302571+0000","last_became_active":"2024-04-26T21:47:15.502923+0000","last_became_peered":"2024-04-26T21:47:15.502923+0000","last_unstale":"2024-04-26T21:47:18.302571+0000","last_undegraded":"2024-04-26T21:47:18.302571+0000","last_fullsized":"2024-04-26T21:47:18.302571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495739+0000","last_change":"2024-04-26T21:47:15.499851+0000","last_active":"2024-04-26T21:47:16.495739+0000","last_peered":"2024-04-26T21:47:16.495739+0000","last_clean":"2024-04-26T21:47:16.495739+0000","last_became_active":"2024-04-26T21:47:15.498066+0000","last_became_peered":"2024-04-26T21:47:15.498066+0000","last_unstale":"2024-04-26T21:47:16.495739+0000","last_undegraded":"2024-04-26T21:47:16.495739+0000","last_fullsized":"2024-04-26T21:47:16.495739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.734287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595590+0000","last_change":"2024-04-26T21:47:15.500779+0000","last_active":"2024-04-26T21:47:16.595590+0000","last_peered":"2024-04-26T21:47:16.595590+0000","last_clean":"2024-04-26T21:47:16.595590+0000","last_became_active":"2024-04-26T21:47:15.500521+0000","last_became_peered":"2024-04-26T21:47:15.500521+0000","last_unstale":"2024-04-26T21:47:16.595590+0000","last_undegraded":"2024-04-26T21:47:16.595590+0000","last_fullsized":"2024-04-26T21:47:16.595590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:21.840377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299815+0000","last_change":"2024-04-26T21:47:15.488048+0000","last_active":"2024-04-26T21:47:18.299815+0000","last_peered":"2024-04-26T21:47:18.299815+0000","last_clean":"2024-04-26T21:47:18.299815+0000","last_became_active":"2024-04-26T21:47:15.487740+0000","last_became_peered":"2024-04-26T21:47:15.487740+0000","last_unstale":"2024-04-26T21:47:18.299815+0000","last_undegraded":"2024-04-26T21:47:18.299815+0000","last_fullsized":"2024-04-26T21:47:18.299815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497735+0000","last_change":"2024-04-26T21:47:15.500182+0000","last_active":"2024-04-26T21:47:16.497735+0000","last_peered":"2024-04-26T21:47:16.497735+0000","last_clean":"2024-04-26T21:47:16.497735+0000","last_became_active":"2024-04-26T21:47:15.499921+0000","last_became_peered":"2024-04-26T21:47:15.499921+0000","last_unstale":"2024-04-26T21:47:16.497735+0000","last_undegraded":"2024-04-26T21:47:16.497735+0000","last_fullsized":"2024-04-26T21:47:16.497735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.734287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498367+0000","last_change":"2024-04-26T21:47:15.499544+0000","last_active":"2024-04-26T21:47:16.498367+0000","last_peered":"2024-04-26T21:47:16.498367+0000","last_clean":"2024-04-26T21:47:16.498367+0000","last_became_active":"2024-04-26T21:47:15.497196+0000","last_became_peered":"2024-04-26T21:47:15.497196+0000","last_unstale":"2024-04-26T21:47:16.498367+0000","last_undegraded":"2024-04-26T21:47:16.498367+0000","last_fullsized":"2024-04-26T21:47:16.498367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302944+0000","last_change":"2024-04-26T21:47:15.503715+0000","last_active":"2024-04-26T21:47:18.302944+0000","last_peered":"2024-04-26T21:47:18.302944+0000","last_clean":"2024-04-26T21:47:18.302944+0000","last_became_active":"2024-04-26T21:47:15.503458+0000","last_became_peered":"2024-04-26T21:47:15.503458+0000","last_unstale":"2024-04-26T21:47:18.302944+0000","last_undegraded":"2024-04-26T21:47:18.302944+0000","last_fullsized":"2024-04-26T21:47:18.302944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:14.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496890+0000","last_change":"2024-04-26T21:47:15.501650+0000","last_active":"2024-04-26T21:47:16.496890+0000","last_peered":"2024-04-26T21:47:16.496890+0000","last_clean":"2024-04-26T21:47:16.496890+0000","last_became_active":"2024-04-26T21:47:15.501307+0000","last_became_peered":"2024-04-26T21:47:15.501307+0000","last_unstale":"2024-04-26T21:47:16.496890+0000","last_undegraded":"2024-04-26T21:47:16.496890+0000","last_fullsized":"2024-04-26T21:47:16.496890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498245+0000","last_change":"2024-04-26T21:47:15.505347+0000","last_active":"2024-04-26T21:47:16.498245+0000","last_peered":"2024-04-26T21:47:16.498245+0000","last_clean":"2024-04-26T21:47:16.498245+0000","last_became_active":"2024-04-26T21:47:15.505054+0000","last_became_peered":"2024-04-26T21:47:15.505054+0000","last_unstale":"2024-04-26T21:47:16.498245+0000","last_undegraded":"2024-04-26T21:47:16.498245+0000","last_fullsized":"2024-04-26T21:47:16.498245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303692+0000","last_change":"2024-04-26T21:47:15.496215+0000","last_active":"2024-04-26T21:47:18.303692+0000","last_peered":"2024-04-26T21:47:18.303692+0000","last_clean":"2024-04-26T21:47:18.303692+0000","last_became_active":"2024-04-26T21:47:15.494518+0000","last_became_peered":"2024-04-26T21:47:15.494518+0000","last_unstale":"2024-04-26T21:47:18.303692+0000","last_undegraded":"2024-04-26T21:47:18.303692+0000","last_fullsized":"2024-04-26T21:47:18.303692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595599+0000","last_change":"2024-04-26T21:47:15.506769+0000","last_active":"2024-04-26T21:47:16.595599+0000","last_peered":"2024-04-26T21:47:16.595599+0000","last_clean":"2024-04-26T21:47:16.595599+0000","last_became_active":"2024-04-26T21:47:15.506081+0000","last_became_peered":"2024-04-26T21:47:15.506081+0000","last_unstale":"2024-04-26T21:47:16.595599+0000","last_undegraded":"2024-04-26T21:47:16.595599+0000","last_fullsized":"2024-04-26T21:47:16.595599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:08.466342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300273+0000","last_change":"2024-04-26T21:47:15.499120+0000","last_active":"2024-04-26T21:47:18.300273+0000","last_peered":"2024-04-26T21:47:18.300273+0000","last_clean":"2024-04-26T21:47:18.300273+0000","last_became_active":"2024-04-26T21:47:15.497893+0000","last_became_peered":"2024-04-26T21:47:15.497893+0000","last_unstale":"2024-04-26T21:47:18.300273+0000","last_undegraded":"2024-04-26T21:47:18.300273+0000","last_fullsized":"2024-04-26T21:47:18.300273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300228+0000","last_change":"2024-04-26T21:47:15.510119+0000","last_active":"2024-04-26T21:47:18.300228+0000","last_peered":"2024-04-26T21:47:18.300228+0000","last_clean":"2024-04-26T21:47:18.300228+0000","last_became_active":"2024-04-26T21:47:15.509832+0000","last_became_peered":"2024-04-26T21:47:15.509832+0000","last_unstale":"2024-04-26T21:47:18.300228+0000","last_undegraded":"2024-04-26T21:47:18.300228+0000","last_fullsized":"2024-04-26T21:47:18.300228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498817+0000","last_change":"2024-04-26T21:47:15.488013+0000","last_active":"2024-04-26T21:47:16.498817+0000","last_peered":"2024-04-26T21:47:16.498817+0000","last_clean":"2024-04-26T21:47:16.498817+0000","last_became_active":"2024-04-26T21:47:15.487642+0000","last_became_peered":"2024-04-26T21:47:15.487642+0000","last_unstale":"2024-04-26T21:47:16.498817+0000","last_undegraded":"2024-04-26T21:47:16.498817+0000","last_fullsized":"2024-04-26T21:47:16.498817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498783+0000","last_change":"2024-04-26T21:47:15.505880+0000","last_active":"2024-04-26T21:47:16.498783+0000","last_peered":"2024-04-26T21:47:16.498783+0000","last_clean":"2024-04-26T21:47:16.498783+0000","last_became_active":"2024-04-26T21:47:15.505570+0000","last_became_peered":"2024-04-26T21:47:15.505570+0000","last_unstale":"2024-04-26T21:47:16.498783+0000","last_undegraded":"2024-04-26T21:47:16.498783+0000","last_fullsized":"2024-04-26T21:47:16.498783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:40.054012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300954+0000","last_change":"2024-04-26T21:47:15.505147+0000","last_active":"2024-04-26T21:47:18.300954+0000","last_peered":"2024-04-26T21:47:18.300954+0000","last_clean":"2024-04-26T21:47:18.300954+0000","last_became_active":"2024-04-26T21:47:15.504613+0000","last_became_peered":"2024-04-26T21:47:15.504613+0000","last_unstale":"2024-04-26T21:47:18.300954+0000","last_undegraded":"2024-04-26T21:47:18.300954+0000","last_fullsized":"2024-04-26T21:47:18.300954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:07.866270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300105+0000","last_change":"2024-04-26T21:47:15.504961+0000","last_active":"2024-04-26T21:47:18.300105+0000","last_peered":"2024-04-26T21:47:18.300105+0000","last_clean":"2024-04-26T21:47:18.300105+0000","last_became_active":"2024-04-26T21:47:15.504583+0000","last_became_peered":"2024-04-26T21:47:15.504583+0000","last_unstale":"2024-04-26T21:47:18.300105+0000","last_undegraded":"2024-04-26T21:47:18.300105+0000","last_fullsized":"2024-04-26T21:47:18.300105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595847+0000","last_change":"2024-04-26T21:47:15.498329+0000","last_active":"2024-04-26T21:47:16.595847+0000","last_peered":"2024-04-26T21:47:16.595847+0000","last_clean":"2024-04-26T21:47:16.595847+0000","last_became_active":"2024-04-26T21:47:15.497328+0000","last_became_peered":"2024-04-26T21:47:15.497328+0000","last_unstale":"2024-04-26T21:47:16.595847+0000","last_undegraded":"2024-04-26T21:47:16.595847+0000","last_fullsized":"2024-04-26T21:47:16.595847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:06.279320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301684+0000","last_change":"2024-04-26T21:47:15.510251+0000","last_active":"2024-04-26T21:47:18.301684+0000","last_peered":"2024-04-26T21:47:18.301684+0000","last_clean":"2024-04-26T21:47:18.301684+0000","last_became_active":"2024-04-26T21:47:15.508589+0000","last_became_peered":"2024-04-26T21:47:15.508589+0000","last_unstale":"2024-04-26T21:47:18.301684+0000","last_undegraded":"2024-04-26T21:47:18.301684+0000","last_fullsized":"2024-04-26T21:47:18.301684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.659159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303727+0000","last_change":"2024-04-26T21:47:15.493959+0000","last_active":"2024-04-26T21:47:18.303727+0000","last_peered":"2024-04-26T21:47:18.303727+0000","last_clean":"2024-04-26T21:47:18.303727+0000","last_became_active":"2024-04-26T21:47:15.493635+0000","last_became_peered":"2024-04-26T21:47:15.493635+0000","last_unstale":"2024-04-26T21:47:18.303727+0000","last_undegraded":"2024-04-26T21:47:18.303727+0000","last_fullsized":"2024-04-26T21:47:18.303727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:30.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498276+0000","last_change":"2024-04-26T21:47:15.501637+0000","last_active":"2024-04-26T21:47:16.498276+0000","last_peered":"2024-04-26T21:47:16.498276+0000","last_clean":"2024-04-26T21:47:16.498276+0000","last_became_active":"2024-04-26T21:47:15.501337+0000","last_became_peered":"2024-04-26T21:47:15.501337+0000","last_unstale":"2024-04-26T21:47:16.498276+0000","last_undegraded":"2024-04-26T21:47:16.498276+0000","last_fullsized":"2024-04-26T21:47:16.498276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:14.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299833+0000","last_change":"2024-04-26T21:47:15.498297+0000","last_active":"2024-04-26T21:47:18.299833+0000","last_peered":"2024-04-26T21:47:18.299833+0000","last_clean":"2024-04-26T21:47:18.299833+0000","last_became_active":"2024-04-26T21:47:15.497962+0000","last_became_peered":"2024-04-26T21:47:15.497962+0000","last_unstale":"2024-04-26T21:47:18.299833+0000","last_undegraded":"2024-04-26T21:47:18.299833+0000","last_fullsized":"2024-04-26T21:47:18.299833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496420+0000","last_change":"2024-04-26T21:47:15.503758+0000","last_active":"2024-04-26T21:47:16.496420+0000","last_peered":"2024-04-26T21:47:16.496420+0000","last_clean":"2024-04-26T21:47:16.496420+0000","last_became_active":"2024-04-26T21:47:15.503053+0000","last_became_peered":"2024-04-26T21:47:15.503053+0000","last_unstale":"2024-04-26T21:47:16.496420+0000","last_undegraded":"2024-04-26T21:47:16.496420+0000","last_fullsized":"2024-04-26T21:47:16.496420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301197+0000","last_change":"2024-04-26T21:47:15.488771+0000","last_active":"2024-04-26T21:47:18.301197+0000","last_peered":"2024-04-26T21:47:18.301197+0000","last_clean":"2024-04-26T21:47:18.301197+0000","last_became_active":"2024-04-26T21:47:15.488378+0000","last_became_peered":"2024-04-26T21:47:15.488378+0000","last_unstale":"2024-04-26T21:47:18.301197+0000","last_undegraded":"2024-04-26T21:47:18.301197+0000","last_fullsized":"2024-04-26T21:47:18.301197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:43.157703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496465+0000","last_change":"2024-04-26T21:47:15.512492+0000","last_active":"2024-04-26T21:47:16.496465+0000","last_peered":"2024-04-26T21:47:16.496465+0000","last_clean":"2024-04-26T21:47:16.496465+0000","last_became_active":"2024-04-26T21:47:15.511942+0000","last_became_peered":"2024-04-26T21:47:15.511942+0000","last_unstale":"2024-04-26T21:47:16.496465+0000","last_undegraded":"2024-04-26T21:47:16.496465+0000","last_fullsized":"2024-04-26T21:47:16.496465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303926+0000","last_change":"2024-04-26T21:47:15.500877+0000","last_active":"2024-04-26T21:47:18.303926+0000","last_peered":"2024-04-26T21:47:18.303926+0000","last_clean":"2024-04-26T21:47:18.303926+0000","last_became_active":"2024-04-26T21:47:15.500565+0000","last_became_peered":"2024-04-26T21:47:15.500565+0000","last_unstale":"2024-04-26T21:47:18.303926+0000","last_undegraded":"2024-04-26T21:47:18.303926+0000","last_fullsized":"2024-04-26T21:47:18.303926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302113+0000","last_change":"2024-04-26T21:47:15.492126+0000","last_active":"2024-04-26T21:47:18.302113+0000","last_peered":"2024-04-26T21:47:18.302113+0000","last_clean":"2024-04-26T21:47:18.302113+0000","last_became_active":"2024-04-26T21:47:15.491855+0000","last_became_peered":"2024-04-26T21:47:15.491855+0000","last_unstale":"2024-04-26T21:47:18.302113+0000","last_undegraded":"2024-04-26T21:47:18.302113+0000","last_fullsized":"2024-04-26T21:47:18.302113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:54.721735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497842+0000","last_change":"2024-04-26T21:47:15.499360+0000","last_active":"2024-04-26T21:47:16.497842+0000","last_peered":"2024-04-26T21:47:16.497842+0000","last_clean":"2024-04-26T21:47:16.497842+0000","last_became_active":"2024-04-26T21:47:15.499193+0000","last_became_peered":"2024-04-26T21:47:15.499193+0000","last_unstale":"2024-04-26T21:47:16.497842+0000","last_undegraded":"2024-04-26T21:47:16.497842+0000","last_fullsized":"2024-04-26T21:47:16.497842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:00.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301308+0000","last_change":"2024-04-26T21:47:15.507840+0000","last_active":"2024-04-26T21:47:18.301308+0000","last_peered":"2024-04-26T21:47:18.301308+0000","last_clean":"2024-04-26T21:47:18.301308+0000","last_became_active":"2024-04-26T21:47:15.507551+0000","last_became_peered":"2024-04-26T21:47:15.507551+0000","last_unstale":"2024-04-26T21:47:18.301308+0000","last_undegraded":"2024-04-26T21:47:18.301308+0000","last_fullsized":"2024-04-26T21:47:18.301308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:32.425606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497923+0000","last_change":"2024-04-26T21:47:15.483176+0000","last_active":"2024-04-26T21:47:16.497923+0000","last_peered":"2024-04-26T21:47:16.497923+0000","last_clean":"2024-04-26T21:47:16.497923+0000","last_became_active":"2024-04-26T21:47:15.481069+0000","last_became_peered":"2024-04-26T21:47:15.481069+0000","last_unstale":"2024-04-26T21:47:16.497923+0000","last_undegraded":"2024-04-26T21:47:16.497923+0000","last_fullsized":"2024-04-26T21:47:16.497923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498984+0000","last_change":"2024-04-26T21:47:15.499599+0000","last_active":"2024-04-26T21:47:16.498984+0000","last_peered":"2024-04-26T21:47:16.498984+0000","last_clean":"2024-04-26T21:47:16.498984+0000","last_became_active":"2024-04-26T21:47:15.499186+0000","last_became_peered":"2024-04-26T21:47:15.499186+0000","last_unstale":"2024-04-26T21:47:16.498984+0000","last_undegraded":"2024-04-26T21:47:16.498984+0000","last_fullsized":"2024-04-26T21:47:16.498984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:03.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303576+0000","last_change":"2024-04-26T21:47:14.483471+0000","last_active":"2024-04-26T21:47:18.303576+0000","last_peered":"2024-04-26T21:47:18.303576+0000","last_clean":"2024-04-26T21:47:18.303576+0000","last_became_active":"2024-04-26T21:47:14.483132+0000","last_became_peered":"2024-04-26T21:47:14.483132+0000","last_unstale":"2024-04-26T21:47:18.303576+0000","last_undegraded":"2024-04-26T21:47:18.303576+0000","last_fullsized":"2024-04-26T21:47:18.303576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:05.265308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303089+0000","last_change":"2024-04-26T21:47:15.499872+0000","last_active":"2024-04-26T21:47:18.303089+0000","last_peered":"2024-04-26T21:47:18.303089+0000","last_clean":"2024-04-26T21:47:18.303089+0000","last_became_active":"2024-04-26T21:47:15.499232+0000","last_became_peered":"2024-04-26T21:47:15.499232+0000","last_unstale":"2024-04-26T21:47:18.303089+0000","last_undegraded":"2024-04-26T21:47:18.303089+0000","last_fullsized":"2024-04-26T21:47:18.303089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:21.539767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499072+0000","last_change":"2024-04-26T21:47:14.477793+0000","last_active":"2024-04-26T21:47:16.499072+0000","last_peered":"2024-04-26T21:47:16.499072+0000","last_clean":"2024-04-26T21:47:16.499072+0000","last_became_active":"2024-04-26T21:47:14.477460+0000","last_became_peered":"2024-04-26T21:47:14.477460+0000","last_unstale":"2024-04-26T21:47:16.499072+0000","last_undegraded":"2024-04-26T21:47:16.499072+0000","last_fullsized":"2024-04-26T21:47:16.499072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303614+0000","last_change":"2024-04-26T21:47:15.495822+0000","last_active":"2024-04-26T21:47:18.303614+0000","last_peered":"2024-04-26T21:47:18.303614+0000","last_clean":"2024-04-26T21:47:18.303614+0000","last_became_active":"2024-04-26T21:47:15.495505+0000","last_became_peered":"2024-04-26T21:47:15.495505+0000","last_unstale":"2024-04-26T21:47:18.303614+0000","last_undegraded":"2024-04-26T21:47:18.303614+0000","last_fullsized":"2024-04-26T21:47:18.303614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:27.340597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300184+0000","last_change":"2024-04-26T21:47:14.484669+0000","last_active":"2024-04-26T21:47:18.300184+0000","last_peered":"2024-04-26T21:47:18.300184+0000","last_clean":"2024-04-26T21:47:18.300184+0000","last_became_active":"2024-04-26T21:47:14.483933+0000","last_became_peered":"2024-04-26T21:47:14.483933+0000","last_unstale":"2024-04-26T21:47:18.300184+0000","last_undegraded":"2024-04-26T21:47:18.300184+0000","last_fullsized":"2024-04-26T21:47:18.300184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496512+0000","last_change":"2024-04-26T21:47:15.512586+0000","last_active":"2024-04-26T21:47:16.496512+0000","last_peered":"2024-04-26T21:47:16.496512+0000","last_clean":"2024-04-26T21:47:16.496512+0000","last_became_active":"2024-04-26T21:47:15.512160+0000","last_became_peered":"2024-04-26T21:47:15.512160+0000","last_unstale":"2024-04-26T21:47:16.496512+0000","last_undegraded":"2024-04-26T21:47:16.496512+0000","last_fullsized":"2024-04-26T21:47:16.496512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595982+0000","last_change":"2024-04-26T21:47:14.462571+0000","last_active":"2024-04-26T21:47:16.595982+0000","last_peered":"2024-04-26T21:47:16.595982+0000","last_clean":"2024-04-26T21:47:16.595982+0000","last_became_active":"2024-04-26T21:47:14.461470+0000","last_became_peered":"2024-04-26T21:47:14.461470+0000","last_unstale":"2024-04-26T21:47:16.595982+0000","last_undegraded":"2024-04-26T21:47:16.595982+0000","last_fullsized":"2024-04-26T21:47:16.595982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:02.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498307+0000","last_change":"2024-04-26T21:47:15.480769+0000","last_active":"2024-04-26T21:47:16.498307+0000","last_peered":"2024-04-26T21:47:16.498307+0000","last_clean":"2024-04-26T21:47:16.498307+0000","last_became_active":"2024-04-26T21:47:15.480389+0000","last_became_peered":"2024-04-26T21:47:15.480389+0000","last_unstale":"2024-04-26T21:47:16.498307+0000","last_undegraded":"2024-04-26T21:47:16.498307+0000","last_fullsized":"2024-04-26T21:47:16.498307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302363+0000","last_change":"2024-04-26T21:47:14.485718+0000","last_active":"2024-04-26T21:47:18.302363+0000","last_peered":"2024-04-26T21:47:18.302363+0000","last_clean":"2024-04-26T21:47:18.302363+0000","last_became_active":"2024-04-26T21:47:14.485458+0000","last_became_peered":"2024-04-26T21:47:14.485458+0000","last_unstale":"2024-04-26T21:47:18.302363+0000","last_undegraded":"2024-04-26T21:47:18.302363+0000","last_fullsized":"2024-04-26T21:47:18.302363+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:29.947919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300532+0000","last_change":"2024-04-26T21:47:15.505533+0000","last_active":"2024-04-26T21:47:18.300532+0000","last_peered":"2024-04-26T21:47:18.300532+0000","last_clean":"2024-04-26T21:47:18.300532+0000","last_became_active":"2024-04-26T21:47:15.505213+0000","last_became_peered":"2024-04-26T21:47:15.505213+0000","last_unstale":"2024-04-26T21:47:18.300532+0000","last_undegraded":"2024-04-26T21:47:18.300532+0000","last_fullsized":"2024-04-26T21:47:18.300532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:52.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303990+0000","last_change":"2024-04-26T21:47:14.484537+0000","last_active":"2024-04-26T21:47:18.303990+0000","last_peered":"2024-04-26T21:47:18.303990+0000","last_clean":"2024-04-26T21:47:18.303990+0000","last_became_active":"2024-04-26T21:47:14.484218+0000","last_became_peered":"2024-04-26T21:47:14.484218+0000","last_unstale":"2024-04-26T21:47:18.303990+0000","last_undegraded":"2024-04-26T21:47:18.303990+0000","last_fullsized":"2024-04-26T21:47:18.303990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:09.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304148+0000","last_change":"2024-04-26T21:47:15.501225+0000","last_active":"2024-04-26T21:47:18.304148+0000","last_peered":"2024-04-26T21:47:18.304148+0000","last_clean":"2024-04-26T21:47:18.304148+0000","last_became_active":"2024-04-26T21:47:15.500826+0000","last_became_peered":"2024-04-26T21:47:15.500826+0000","last_unstale":"2024-04-26T21:47:18.304148+0000","last_undegraded":"2024-04-26T21:47:18.304148+0000","last_fullsized":"2024-04-26T21:47:18.304148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:03.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299589+0000","last_change":"2024-04-26T21:47:14.483496+0000","last_active":"2024-04-26T21:47:18.299589+0000","last_peered":"2024-04-26T21:47:18.299589+0000","last_clean":"2024-04-26T21:47:18.299589+0000","last_became_active":"2024-04-26T21:47:14.483201+0000","last_became_peered":"2024-04-26T21:47:14.483201+0000","last_unstale":"2024-04-26T21:47:18.299589+0000","last_undegraded":"2024-04-26T21:47:18.299589+0000","last_fullsized":"2024-04-26T21:47:18.299589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495421+0000","last_change":"2024-04-26T21:47:15.506056+0000","last_active":"2024-04-26T21:47:16.495421+0000","last_peered":"2024-04-26T21:47:16.495421+0000","last_clean":"2024-04-26T21:47:16.495421+0000","last_became_active":"2024-04-26T21:47:15.505645+0000","last_became_peered":"2024-04-26T21:47:15.505645+0000","last_unstale":"2024-04-26T21:47:16.495421+0000","last_undegraded":"2024-04-26T21:47:16.495421+0000","last_fullsized":"2024-04-26T21:47:16.495421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:17.697309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304021+0000","last_change":"2024-04-26T21:47:14.480135+0000","last_active":"2024-04-26T21:47:18.304021+0000","last_peered":"2024-04-26T21:47:18.304021+0000","last_clean":"2024-04-26T21:47:18.304021+0000","last_became_active":"2024-04-26T21:47:14.479762+0000","last_became_peered":"2024-04-26T21:47:14.479762+0000","last_unstale":"2024-04-26T21:47:18.304021+0000","last_undegraded":"2024-04-26T21:47:18.304021+0000","last_fullsized":"2024-04-26T21:47:18.304021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:00.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300813+0000","last_change":"2024-04-26T21:47:15.498265+0000","last_active":"2024-04-26T21:47:18.300813+0000","last_peered":"2024-04-26T21:47:18.300813+0000","last_clean":"2024-04-26T21:47:18.300813+0000","last_became_active":"2024-04-26T21:47:15.497885+0000","last_became_peered":"2024-04-26T21:47:15.497885+0000","last_unstale":"2024-04-26T21:47:18.300813+0000","last_undegraded":"2024-04-26T21:47:18.300813+0000","last_fullsized":"2024-04-26T21:47:18.300813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:14.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301177+0000","last_change":"2024-04-26T21:47:14.484137+0000","last_active":"2024-04-26T21:47:18.301177+0000","last_peered":"2024-04-26T21:47:18.301177+0000","last_clean":"2024-04-26T21:47:18.301177+0000","last_became_active":"2024-04-26T21:47:14.483547+0000","last_became_peered":"2024-04-26T21:47:14.483547+0000","last_unstale":"2024-04-26T21:47:18.301177+0000","last_undegraded":"2024-04-26T21:47:18.301177+0000","last_fullsized":"2024-04-26T21:47:18.301177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:26.685031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301124+0000","last_change":"2024-04-26T21:47:15.506733+0000","last_active":"2024-04-26T21:47:18.301124+0000","last_peered":"2024-04-26T21:47:18.301124+0000","last_clean":"2024-04-26T21:47:18.301124+0000","last_became_active":"2024-04-26T21:47:15.506287+0000","last_became_peered":"2024-04-26T21:47:15.506287+0000","last_unstale":"2024-04-26T21:47:18.301124+0000","last_undegraded":"2024-04-26T21:47:18.301124+0000","last_fullsized":"2024-04-26T21:47:18.301124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:09.857376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499211+0000","last_change":"2024-04-26T21:47:14.485542+0000","last_active":"2024-04-26T21:47:16.499211+0000","last_peered":"2024-04-26T21:47:16.499211+0000","last_clean":"2024-04-26T21:47:16.499211+0000","last_became_active":"2024-04-26T21:47:14.485231+0000","last_became_peered":"2024-04-26T21:47:14.485231+0000","last_unstale":"2024-04-26T21:47:16.499211+0000","last_undegraded":"2024-04-26T21:47:16.499211+0000","last_fullsized":"2024-04-26T21:47:16.499211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300552+0000","last_change":"2024-04-26T21:47:15.486843+0000","last_active":"2024-04-26T21:47:18.300552+0000","last_peered":"2024-04-26T21:47:18.300552+0000","last_clean":"2024-04-26T21:47:18.300552+0000","last_became_active":"2024-04-26T21:47:15.486560+0000","last_became_peered":"2024-04-26T21:47:15.486560+0000","last_unstale":"2024-04-26T21:47:18.300552+0000","last_undegraded":"2024-04-26T21:47:18.300552+0000","last_fullsized":"2024-04-26T21:47:18.300552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:28.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302825+0000","last_change":"2024-04-26T21:47:14.472755+0000","last_active":"2024-04-26T21:47:18.302825+0000","last_peered":"2024-04-26T21:47:18.302825+0000","last_clean":"2024-04-26T21:47:18.302825+0000","last_became_active":"2024-04-26T21:47:14.472547+0000","last_became_peered":"2024-04-26T21:47:14.472547+0000","last_unstale":"2024-04-26T21:47:18.302825+0000","last_undegraded":"2024-04-26T21:47:18.302825+0000","last_fullsized":"2024-04-26T21:47:18.302825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302172+0000","last_change":"2024-04-26T21:47:15.507039+0000","last_active":"2024-04-26T21:47:18.302172+0000","last_peered":"2024-04-26T21:47:18.302172+0000","last_clean":"2024-04-26T21:47:18.302172+0000","last_became_active":"2024-04-26T21:47:15.503211+0000","last_became_peered":"2024-04-26T21:47:15.503211+0000","last_unstale":"2024-04-26T21:47:18.302172+0000","last_undegraded":"2024-04-26T21:47:18.302172+0000","last_fullsized":"2024-04-26T21:47:18.302172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300534+0000","last_change":"2024-04-26T21:47:14.487095+0000","last_active":"2024-04-26T21:47:18.300534+0000","last_peered":"2024-04-26T21:47:18.300534+0000","last_clean":"2024-04-26T21:47:18.300534+0000","last_became_active":"2024-04-26T21:47:14.486802+0000","last_became_peered":"2024-04-26T21:47:14.486802+0000","last_unstale":"2024-04-26T21:47:18.300534+0000","last_undegraded":"2024-04-26T21:47:18.300534+0000","last_fullsized":"2024-04-26T21:47:18.300534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.645147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495499+0000","last_change":"2024-04-26T21:47:15.505778+0000","last_active":"2024-04-26T21:47:16.495499+0000","last_peered":"2024-04-26T21:47:16.495499+0000","last_clean":"2024-04-26T21:47:16.495499+0000","last_became_active":"2024-04-26T21:47:15.505505+0000","last_became_peered":"2024-04-26T21:47:15.505505+0000","last_unstale":"2024-04-26T21:47:16.495499+0000","last_undegraded":"2024-04-26T21:47:16.495499+0000","last_fullsized":"2024-04-26T21:47:16.495499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:13.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302680+0000","last_change":"2024-04-26T21:47:14.478529+0000","last_active":"2024-04-26T21:47:18.302680+0000","last_peered":"2024-04-26T21:47:18.302680+0000","last_clean":"2024-04-26T21:47:18.302680+0000","last_became_active":"2024-04-26T21:47:14.478281+0000","last_became_peered":"2024-04-26T21:47:14.478281+0000","last_unstale":"2024-04-26T21:47:18.302680+0000","last_undegraded":"2024-04-26T21:47:18.302680+0000","last_fullsized":"2024-04-26T21:47:18.302680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.300428+0000","last_change":"2024-04-26T21:47:15.506746+0000","last_active":"2024-04-26T21:47:18.300428+0000","last_peered":"2024-04-26T21:47:18.300428+0000","last_clean":"2024-04-26T21:47:18.300428+0000","last_became_active":"2024-04-26T21:47:15.506474+0000","last_became_peered":"2024-04-26T21:47:15.506474+0000","last_unstale":"2024-04-26T21:47:18.300428+0000","last_undegraded":"2024-04-26T21:47:18.300428+0000","last_fullsized":"2024-04-26T21:47:18.300428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:05.016587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495576+0000","last_change":"2024-04-26T21:47:14.480124+0000","last_active":"2024-04-26T21:47:16.495576+0000","last_peered":"2024-04-26T21:47:16.495576+0000","last_clean":"2024-04-26T21:47:16.495576+0000","last_became_active":"2024-04-26T21:47:14.479846+0000","last_became_peered":"2024-04-26T21:47:14.479846+0000","last_unstale":"2024-04-26T21:47:16.495576+0000","last_undegraded":"2024-04-26T21:47:16.495576+0000","last_fullsized":"2024-04-26T21:47:16.495576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:52.748418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498481+0000","last_change":"2024-04-26T21:47:15.482410+0000","last_active":"2024-04-26T21:47:16.498481+0000","last_peered":"2024-04-26T21:47:16.498481+0000","last_clean":"2024-04-26T21:47:16.498481+0000","last_became_active":"2024-04-26T21:47:15.480603+0000","last_became_peered":"2024-04-26T21:47:15.480603+0000","last_unstale":"2024-04-26T21:47:16.498481+0000","last_undegraded":"2024-04-26T21:47:16.498481+0000","last_fullsized":"2024-04-26T21:47:16.498481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:21.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299325+0000","last_change":"2024-04-26T21:47:14.485004+0000","last_active":"2024-04-26T21:47:18.299325+0000","last_peered":"2024-04-26T21:47:18.299325+0000","last_clean":"2024-04-26T21:47:18.299325+0000","last_became_active":"2024-04-26T21:47:14.483355+0000","last_became_peered":"2024-04-26T21:47:14.483355+0000","last_unstale":"2024-04-26T21:47:18.299325+0000","last_undegraded":"2024-04-26T21:47:18.299325+0000","last_fullsized":"2024-04-26T21:47:18.299325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301721+0000","last_change":"2024-04-26T21:47:15.511291+0000","last_active":"2024-04-26T21:47:18.301721+0000","last_peered":"2024-04-26T21:47:18.301721+0000","last_clean":"2024-04-26T21:47:18.301721+0000","last_became_active":"2024-04-26T21:47:15.509717+0000","last_became_peered":"2024-04-26T21:47:15.509717+0000","last_unstale":"2024-04-26T21:47:18.301721+0000","last_undegraded":"2024-04-26T21:47:18.301721+0000","last_fullsized":"2024-04-26T21:47:18.301721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.659159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299751+0000","last_change":"2024-04-26T21:47:14.482285+0000","last_active":"2024-04-26T21:47:18.299751+0000","last_peered":"2024-04-26T21:47:18.299751+0000","last_clean":"2024-04-26T21:47:18.299751+0000","last_became_active":"2024-04-26T21:47:14.481952+0000","last_became_peered":"2024-04-26T21:47:14.481952+0000","last_unstale":"2024-04-26T21:47:18.299751+0000","last_undegraded":"2024-04-26T21:47:18.299751+0000","last_fullsized":"2024-04-26T21:47:18.299751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:53.707723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301622+0000","last_change":"2024-04-26T21:47:15.486505+0000","last_active":"2024-04-26T21:47:18.301622+0000","last_peered":"2024-04-26T21:47:18.301622+0000","last_clean":"2024-04-26T21:47:18.301622+0000","last_became_active":"2024-04-26T21:47:15.486070+0000","last_became_peered":"2024-04-26T21:47:15.486070+0000","last_unstale":"2024-04-26T21:47:18.301622+0000","last_undegraded":"2024-04-26T21:47:18.301622+0000","last_fullsized":"2024-04-26T21:47:18.301622+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:21.539767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496639+0000","last_change":"2024-04-26T21:47:14.485819+0000","last_active":"2024-04-26T21:47:16.496639+0000","last_peered":"2024-04-26T21:47:16.496639+0000","last_clean":"2024-04-26T21:47:16.496639+0000","last_became_active":"2024-04-26T21:47:14.485435+0000","last_became_peered":"2024-04-26T21:47:14.485435+0000","last_unstale":"2024-04-26T21:47:16.496639+0000","last_undegraded":"2024-04-26T21:47:16.496639+0000","last_fullsized":"2024-04-26T21:47:16.496639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301477+0000","last_change":"2024-04-26T21:47:15.489120+0000","last_active":"2024-04-26T21:47:18.301477+0000","last_peered":"2024-04-26T21:47:18.301477+0000","last_clean":"2024-04-26T21:47:18.301477+0000","last_became_active":"2024-04-26T21:47:15.488799+0000","last_became_peered":"2024-04-26T21:47:15.488799+0000","last_unstale":"2024-04-26T21:47:18.301477+0000","last_undegraded":"2024-04-26T21:47:18.301477+0000","last_fullsized":"2024-04-26T21:47:18.301477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301934+0000","last_change":"2024-04-26T21:47:14.481662+0000","last_active":"2024-04-26T21:47:18.301934+0000","last_peered":"2024-04-26T21:47:18.301934+0000","last_clean":"2024-04-26T21:47:18.301934+0000","last_became_active":"2024-04-26T21:47:14.480582+0000","last_became_peered":"2024-04-26T21:47:14.480582+0000","last_unstale":"2024-04-26T21:47:18.301934+0000","last_undegraded":"2024-04-26T21:47:18.301934+0000","last_fullsized":"2024-04-26T21:47:18.301934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:07.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496000+0000","last_change":"2024-04-26T21:47:15.499588+0000","last_active":"2024-04-26T21:47:16.496000+0000","last_peered":"2024-04-26T21:47:16.496000+0000","last_clean":"2024-04-26T21:47:16.496000+0000","last_became_active":"2024-04-26T21:47:15.496595+0000","last_became_peered":"2024-04-26T21:47:15.496595+0000","last_unstale":"2024-04-26T21:47:16.496000+0000","last_undegraded":"2024-04-26T21:47:16.496000+0000","last_fullsized":"2024-04-26T21:47:16.496000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.304442+0000","last_change":"2024-04-26T21:47:14.475423+0000","last_active":"2024-04-26T21:47:18.304442+0000","last_peered":"2024-04-26T21:47:18.304442+0000","last_clean":"2024-04-26T21:47:18.304442+0000","last_became_active":"2024-04-26T21:47:14.475083+0000","last_became_peered":"2024-04-26T21:47:14.475083+0000","last_unstale":"2024-04-26T21:47:18.304442+0000","last_undegraded":"2024-04-26T21:47:18.304442+0000","last_fullsized":"2024-04-26T21:47:18.304442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:20.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.596005+0000","last_change":"2024-04-26T21:47:15.499100+0000","last_active":"2024-04-26T21:47:16.596005+0000","last_peered":"2024-04-26T21:47:16.596005+0000","last_clean":"2024-04-26T21:47:16.596005+0000","last_became_active":"2024-04-26T21:47:15.498023+0000","last_became_peered":"2024-04-26T21:47:15.498023+0000","last_unstale":"2024-04-26T21:47:16.596005+0000","last_undegraded":"2024-04-26T21:47:16.596005+0000","last_fullsized":"2024-04-26T21:47:16.596005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:55.763012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.303875+0000","last_change":"2024-04-26T21:47:14.487648+0000","last_active":"2024-04-26T21:47:18.303875+0000","last_peered":"2024-04-26T21:47:18.303875+0000","last_clean":"2024-04-26T21:47:18.303875+0000","last_became_active":"2024-04-26T21:47:14.487356+0000","last_became_peered":"2024-04-26T21:47:14.487356+0000","last_unstale":"2024-04-26T21:47:18.303875+0000","last_undegraded":"2024-04-26T21:47:18.303875+0000","last_fullsized":"2024-04-26T21:47:18.303875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:15.031293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595646+0000","last_change":"2024-04-26T21:47:15.503358+0000","last_active":"2024-04-26T21:47:16.595646+0000","last_peered":"2024-04-26T21:47:16.595646+0000","last_clean":"2024-04-26T21:47:16.595646+0000","last_became_active":"2024-04-26T21:47:15.502980+0000","last_became_peered":"2024-04-26T21:47:15.502980+0000","last_unstale":"2024-04-26T21:47:16.595646+0000","last_undegraded":"2024-04-26T21:47:16.595646+0000","last_fullsized":"2024-04-26T21:47:16.595646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:30.961931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498624+0000","last_change":"2024-04-26T21:47:14.478983+0000","last_active":"2024-04-26T21:47:16.498624+0000","last_peered":"2024-04-26T21:47:16.498624+0000","last_clean":"2024-04-26T21:47:16.498624+0000","last_became_active":"2024-04-26T21:47:14.478698+0000","last_became_peered":"2024-04-26T21:47:14.478698+0000","last_unstale":"2024-04-26T21:47:16.498624+0000","last_undegraded":"2024-04-26T21:47:16.498624+0000","last_fullsized":"2024-04-26T21:47:16.498624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595732+0000","last_change":"2024-04-26T21:47:15.507004+0000","last_active":"2024-04-26T21:47:16.595732+0000","last_peered":"2024-04-26T21:47:16.595732+0000","last_clean":"2024-04-26T21:47:16.595732+0000","last_became_active":"2024-04-26T21:47:15.506486+0000","last_became_peered":"2024-04-26T21:47:15.506486+0000","last_unstale":"2024-04-26T21:47:16.595732+0000","last_undegraded":"2024-04-26T21:47:16.595732+0000","last_fullsized":"2024-04-26T21:47:16.595732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:21.290574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302825+0000","last_change":"2024-04-26T21:47:14.482597+0000","last_active":"2024-04-26T21:47:18.302825+0000","last_peered":"2024-04-26T21:47:18.302825+0000","last_clean":"2024-04-26T21:47:18.302825+0000","last_became_active":"2024-04-26T21:47:14.482299+0000","last_became_peered":"2024-04-26T21:47:14.482299+0000","last_unstale":"2024-04-26T21:47:18.302825+0000","last_undegraded":"2024-04-26T21:47:18.302825+0000","last_fullsized":"2024-04-26T21:47:18.302825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:20.276562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595420+0000","last_change":"2024-04-26T21:47:15.505684+0000","last_active":"2024-04-26T21:47:16.595420+0000","last_peered":"2024-04-26T21:47:16.595420+0000","last_clean":"2024-04-26T21:47:16.595420+0000","last_became_active":"2024-04-26T21:47:15.505416+0000","last_became_peered":"2024-04-26T21:47:15.505416+0000","last_unstale":"2024-04-26T21:47:16.595420+0000","last_undegraded":"2024-04-26T21:47:16.595420+0000","last_fullsized":"2024-04-26T21:47:16.595420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:10.577425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301724+0000","last_change":"2024-04-26T21:47:14.475056+0000","last_active":"2024-04-26T21:47:18.301724+0000","last_peered":"2024-04-26T21:47:18.301724+0000","last_clean":"2024-04-26T21:47:18.301724+0000","last_became_active":"2024-04-26T21:47:14.474735+0000","last_became_peered":"2024-04-26T21:47:14.474735+0000","last_unstale":"2024-04-26T21:47:18.301724+0000","last_undegraded":"2024-04-26T21:47:18.301724+0000","last_fullsized":"2024-04-26T21:47:18.301724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:58.661753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497558+0000","last_change":"2024-04-26T21:47:15.509371+0000","last_active":"2024-04-26T21:47:16.497558+0000","last_peered":"2024-04-26T21:47:16.497558+0000","last_clean":"2024-04-26T21:47:16.497558+0000","last_became_active":"2024-04-26T21:47:15.508964+0000","last_became_peered":"2024-04-26T21:47:15.508964+0000","last_unstale":"2024-04-26T21:47:16.497558+0000","last_undegraded":"2024-04-26T21:47:16.497558+0000","last_fullsized":"2024-04-26T21:47:16.497558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:13.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299714+0000","last_change":"2024-04-26T21:47:14.478761+0000","last_active":"2024-04-26T21:47:18.299714+0000","last_peered":"2024-04-26T21:47:18.299714+0000","last_clean":"2024-04-26T21:47:18.299714+0000","last_became_active":"2024-04-26T21:47:14.477387+0000","last_became_peered":"2024-04-26T21:47:14.477387+0000","last_unstale":"2024-04-26T21:47:18.299714+0000","last_undegraded":"2024-04-26T21:47:18.299714+0000","last_fullsized":"2024-04-26T21:47:18.299714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:51.892485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.595837+0000","last_change":"2024-04-26T21:47:15.497721+0000","last_active":"2024-04-26T21:47:16.595837+0000","last_peered":"2024-04-26T21:47:16.595837+0000","last_clean":"2024-04-26T21:47:16.595837+0000","last_became_active":"2024-04-26T21:47:15.497389+0000","last_became_peered":"2024-04-26T21:47:15.497389+0000","last_unstale":"2024-04-26T21:47:16.595837+0000","last_undegraded":"2024-04-26T21:47:16.595837+0000","last_fullsized":"2024-04-26T21:47:16.595837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:42.246393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496744+0000","last_change":"2024-04-26T21:47:14.476244+0000","last_active":"2024-04-26T21:47:16.496744+0000","last_peered":"2024-04-26T21:47:16.496744+0000","last_clean":"2024-04-26T21:47:16.496744+0000","last_became_active":"2024-04-26T21:47:14.475929+0000","last_became_peered":"2024-04-26T21:47:14.475929+0000","last_unstale":"2024-04-26T21:47:16.496744+0000","last_undegraded":"2024-04-26T21:47:16.496744+0000","last_fullsized":"2024-04-26T21:47:16.496744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301935+0000","last_change":"2024-04-26T21:47:15.505586+0000","last_active":"2024-04-26T21:47:18.301935+0000","last_peered":"2024-04-26T21:47:18.301935+0000","last_clean":"2024-04-26T21:47:18.301935+0000","last_became_active":"2024-04-26T21:47:15.505306+0000","last_became_peered":"2024-04-26T21:47:15.505306+0000","last_unstale":"2024-04-26T21:47:18.301935+0000","last_undegraded":"2024-04-26T21:47:18.301935+0000","last_fullsized":"2024-04-26T21:47:18.301935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:48.488770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.499190+0000","last_change":"2024-04-26T21:47:14.479190+0000","last_active":"2024-04-26T21:47:16.499190+0000","last_peered":"2024-04-26T21:47:16.499190+0000","last_clean":"2024-04-26T21:47:16.499190+0000","last_became_active":"2024-04-26T21:47:14.478861+0000","last_became_peered":"2024-04-26T21:47:14.478861+0000","last_unstale":"2024-04-26T21:47:16.499190+0000","last_undegraded":"2024-04-26T21:47:16.499190+0000","last_fullsized":"2024-04-26T21:47:16.499190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498740+0000","last_change":"2024-04-26T21:47:15.507270+0000","last_active":"2024-04-26T21:47:16.498740+0000","last_peered":"2024-04-26T21:47:16.498740+0000","last_clean":"2024-04-26T21:47:16.498740+0000","last_became_active":"2024-04-26T21:47:15.505462+0000","last_became_peered":"2024-04-26T21:47:15.505462+0000","last_unstale":"2024-04-26T21:47:16.498740+0000","last_undegraded":"2024-04-26T21:47:16.498740+0000","last_fullsized":"2024-04-26T21:47:16.498740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:27.340597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.496078+0000","last_change":"2024-04-26T21:47:14.465459+0000","last_active":"2024-04-26T21:47:16.496078+0000","last_peered":"2024-04-26T21:47:16.496078+0000","last_clean":"2024-04-26T21:47:16.496078+0000","last_became_active":"2024-04-26T21:47:14.464667+0000","last_became_peered":"2024-04-26T21:47:14.464667+0000","last_unstale":"2024-04-26T21:47:16.496078+0000","last_undegraded":"2024-04-26T21:47:16.496078+0000","last_fullsized":"2024-04-26T21:47:16.496078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:20.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.302583+0000","last_change":"2024-04-26T21:47:15.506707+0000","last_active":"2024-04-26T21:47:18.302583+0000","last_peered":"2024-04-26T21:47:18.302583+0000","last_clean":"2024-04-26T21:47:18.302583+0000","last_became_active":"2024-04-26T21:47:15.506405+0000","last_became_peered":"2024-04-26T21:47:15.506405+0000","last_unstale":"2024-04-26T21:47:18.302583+0000","last_undegraded":"2024-04-26T21:47:18.302583+0000","last_fullsized":"2024-04-26T21:47:18.302583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.906497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498915+0000","last_change":"2024-04-26T21:47:14.481643+0000","last_active":"2024-04-26T21:47:16.498915+0000","last_peered":"2024-04-26T21:47:16.498915+0000","last_clean":"2024-04-26T21:47:16.498915+0000","last_became_active":"2024-04-26T21:47:14.481119+0000","last_became_peered":"2024-04-26T21:47:14.481119+0000","last_unstale":"2024-04-26T21:47:16.498915+0000","last_undegraded":"2024-04-26T21:47:16.498915+0000","last_fullsized":"2024-04-26T21:47:16.498915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:31.411594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498754+0000","last_change":"2024-04-26T21:47:15.511552+0000","last_active":"2024-04-26T21:47:16.498754+0000","last_peered":"2024-04-26T21:47:16.498754+0000","last_clean":"2024-04-26T21:47:16.498754+0000","last_became_active":"2024-04-26T21:47:15.511178+0000","last_became_peered":"2024-04-26T21:47:15.511178+0000","last_unstale":"2024-04-26T21:47:16.498754+0000","last_undegraded":"2024-04-26T21:47:16.498754+0000","last_fullsized":"2024-04-26T21:47:16.498754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:30.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.495610+0000","last_change":"2024-04-26T21:47:14.475520+0000","last_active":"2024-04-26T21:47:16.495610+0000","last_peered":"2024-04-26T21:47:16.495610+0000","last_clean":"2024-04-26T21:47:16.495610+0000","last_became_active":"2024-04-26T21:47:14.475208+0000","last_became_peered":"2024-04-26T21:47:14.475208+0000","last_unstale":"2024-04-26T21:47:16.495610+0000","last_undegraded":"2024-04-26T21:47:16.495610+0000","last_fullsized":"2024-04-26T21:47:16.495610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:26.837585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299460+0000","last_change":"2024-04-26T21:47:15.493968+0000","last_active":"2024-04-26T21:47:18.299460+0000","last_peered":"2024-04-26T21:47:18.299460+0000","last_clean":"2024-04-26T21:47:18.299460+0000","last_became_active":"2024-04-26T21:47:15.493681+0000","last_became_peered":"2024-04-26T21:47:15.493681+0000","last_unstale":"2024-04-26T21:47:18.299460+0000","last_undegraded":"2024-04-26T21:47:18.299460+0000","last_fullsized":"2024-04-26T21:47:18.299460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:46.021803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301888+0000","last_change":"2024-04-26T21:47:14.477761+0000","last_active":"2024-04-26T21:47:18.301888+0000","last_peered":"2024-04-26T21:47:18.301888+0000","last_clean":"2024-04-26T21:47:18.301888+0000","last_became_active":"2024-04-26T21:47:14.477292+0000","last_became_peered":"2024-04-26T21:47:14.477292+0000","last_unstale":"2024-04-26T21:47:18.301888+0000","last_undegraded":"2024-04-26T21:47:18.301888+0000","last_fullsized":"2024-04-26T21:47:18.301888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:20.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301469+0000","last_change":"2024-04-26T21:47:15.494026+0000","last_active":"2024-04-26T21:47:18.301469+0000","last_peered":"2024-04-26T21:47:18.301469+0000","last_clean":"2024-04-26T21:47:18.301469+0000","last_became_active":"2024-04-26T21:47:15.493682+0000","last_became_peered":"2024-04-26T21:47:15.493682+0000","last_unstale":"2024-04-26T21:47:18.301469+0000","last_undegraded":"2024-04-26T21:47:18.301469+0000","last_fullsized":"2024-04-26T21:47:18.301469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:27.699043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.299529+0000","last_change":"2024-04-26T21:47:14.485177+0000","last_active":"2024-04-26T21:47:18.299529+0000","last_peered":"2024-04-26T21:47:18.299529+0000","last_clean":"2024-04-26T21:47:18.299529+0000","last_became_active":"2024-04-26T21:47:14.483874+0000","last_became_peered":"2024-04-26T21:47:14.483874+0000","last_unstale":"2024-04-26T21:47:18.299529+0000","last_undegraded":"2024-04-26T21:47:18.299529+0000","last_fullsized":"2024-04-26T21:47:18.299529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:38.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.497850+0000","last_change":"2024-04-26T21:47:15.497283+0000","last_active":"2024-04-26T21:47:16.497850+0000","last_peered":"2024-04-26T21:47:16.497850+0000","last_clean":"2024-04-26T21:47:16.497850+0000","last_became_active":"2024-04-26T21:47:15.497045+0000","last_became_peered":"2024-04-26T21:47:15.497045+0000","last_unstale":"2024-04-26T21:47:16.497850+0000","last_undegraded":"2024-04-26T21:47:16.497850+0000","last_fullsized":"2024-04-26T21:47:16.497850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:14.434114+0000","last_clean_scrub_stamp":"2024-04-26T21:47:14.434114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:07.193990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:18.301713+0000","last_change":"2024-04-26T21:47:14.480541+0000","last_active":"2024-04-26T21:47:18.301713+0000","last_peered":"2024-04-26T21:47:18.301713+0000","last_clean":"2024-04-26T21:47:18.301713+0000","last_became_active":"2024-04-26T21:47:14.480243+0000","last_became_peered":"2024-04-26T21:47:14.480243+0000","last_unstale":"2024-04-26T21:47:18.301713+0000","last_undegraded":"2024-04-26T21:47:18.301713+0000","last_fullsized":"2024-04-26T21:47:18.301713+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:13.420102+0000","last_clean_scrub_stamp":"2024-04-26T21:47:13.420102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:41.232380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:47:16.498753+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:16.498753+0000","last_peered":"2024-04-26T21:47:16.498753+0000","last_clean":"2024-04-26T21:47:16.498753+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:16.498753+0000","last_undegraded":"2024-04-26T21:47:16.498753+0000","last_fullsized":"2024-04-26T21:47:16.498753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:25.343 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:47:25.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:47:25.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.595+0000 7f70d1a00640 1 Processor -- start 2024-04-26T21:47:25.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.595+0000 7f70d1a00640 1 -- start start 2024-04-26T21:47:25.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.597+0000 7f70d1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f70cc06f450 0x7f70cc0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.597+0000 7f70d1a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70cc15fe20 0x7f70cc0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.597+0000 7f70d1a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70cc0b5180 con 0x7f70cc0b5560 2024-04-26T21:47:25.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.597+0000 7f70d1a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f70cc1602f0 con 0x7f70cc15fe20 2024-04-26T21:47:25.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.597+0000 7f70d1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f70cc1605d0 con 0x7f70cc06f450 2024-04-26T21:47:25.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70c3e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f70cc0b5560 0x7f70cc15f710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43196/0 (socket says 172.21.15.106:43196) 2024-04-26T21:47:25.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70c3e00640 1 -- 172.21.15.106:0/1088368030 learned_addr learned my addr 172.21.15.106:0/1088368030 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:25.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70cb400640 1 --2- 172.21.15.106:0/1088368030 >> v2:172.21.15.106:3300/0 conn(0x7f70cc15fe20 0x7f70cc0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70caa00640 1 --2- 172.21.15.106:0/1088368030 >> v2:172.21.15.204:3300/0 conn(0x7f70cc06f450 0x7f70cc0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70ca000640 1 -- 172.21.15.106:0/1088368030 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063429687 0 0) 0x7f70cc0b5180 con 0x7f70cc0b5560 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70ca000640 1 -- 172.21.15.106:0/1088368030 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70b0003520 con 0x7f70cc0b5560 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70cb400640 1 -- 172.21.15.106:0/1088368030 >> v2:172.21.15.204:3300/0 conn(0x7f70cc06f450 msgr2=0x7f70cc0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70cb400640 1 --2- 172.21.15.106:0/1088368030 >> v2:172.21.15.204:3300/0 conn(0x7f70cc06f450 0x7f70cc0b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70cb400640 1 -- 172.21.15.106:0/1088368030 >> v1:172.21.15.106:6789/0 conn(0x7f70cc0b5560 legacy=0x7f70cc15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.598+0000 7f70cb400640 1 -- 172.21.15.106:0/1088368030 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc0b4160 con 0x7f70cc15fe20 2024-04-26T21:47:25.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.599+0000 7f70cb400640 1 --2- 172.21.15.106:0/1088368030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc15fe20 0x7f70cc0b59e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f70cc073290 tx=0x7f70c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6125e5c079875564 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70ca000640 1 -- 172.21.15.106:0/1088368030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4007740 con 0x7f70cc15fe20 2024-04-26T21:47:25.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70ca000640 1 -- 172.21.15.106:0/1088368030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c40078e0 con 0x7f70cc15fe20 2024-04-26T21:47:25.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70ca000640 1 -- 172.21.15.106:0/1088368030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4007cb0 con 0x7f70cc15fe20 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 -- 172.21.15.106:0/1088368030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc15fe20 msgr2=0x7f70cc0b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 --2- 172.21.15.106:0/1088368030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc15fe20 0x7f70cc0b59e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f70cc073290 tx=0x7f70c402a680 comp rx=0 tx=0).stop 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 -- 172.21.15.106:0/1088368030 shutdown_connections 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 --2- 172.21.15.106:0/1088368030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc15fe20 0x7f70cc0b59e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 --2- 172.21.15.106:0/1088368030 >> v2:172.21.15.204:3300/0 conn(0x7f70cc06f450 0x7f70cc0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 -- 172.21.15.106:0/1088368030 >> 172.21.15.106:0/1088368030 conn(0x7f70cc0b2a30 msgr2=0x7f70cc0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 -- 172.21.15.106:0/1088368030 shutdown_connections 2024-04-26T21:47:25.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.600+0000 7f70d1a00640 1 -- 172.21.15.106:0/1088368030 wait complete. 2024-04-26T21:47:25.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.602+0000 7f70d1a00640 1 Processor -- start 2024-04-26T21:47:25.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.602+0000 7f70d1a00640 1 -- start start 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70cc06f450 0x7f70cc15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70cc15fe20 0x7f70cc09e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc06dde0 con 0x7f70cc0b5560 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc074160 con 0x7f70cc15fe20 2024-04-26T21:47:25.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70d1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc06df60 con 0x7f70cc06f450 2024-04-26T21:47:25.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70c3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70c3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53604/0 (socket says 172.21.15.106:53604) 2024-04-26T21:47:25.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.603+0000 7f70c3e00640 1 -- 172.21.15.106:0/3523685145 learned_addr learned my addr 172.21.15.106:0/3523685145 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70cb400640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70cc15fe20 0x7f70cc09e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 -- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70cc06f450 msgr2=0x7f70cc15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70cc06f450 0x7f70cc15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 -- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70cc15fe20 msgr2=0x7f70cc09e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70cc15fe20 0x7f70cc09e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 -- 172.21.15.106:0/3523685145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc09bbd0 con 0x7f70cc0b5560 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c3e00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c400 tx=0x7f70b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4017070 con 0x7f70cc0b5560 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4005d00 con 0x7f70cc0b5560 2024-04-26T21:47:25.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.604+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4007310 con 0x7f70cc0b5560 2024-04-26T21:47:25.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.605+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70cc09beb0 con 0x7f70cc0b5560 2024-04-26T21:47:25.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.605+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70cc15eea0 con 0x7f70cc0b5560 2024-04-26T21:47:25.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.606+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70cc069650 con 0x7f70cc0b5560 2024-04-26T21:47:25.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.607+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70b401a070 con 0x7f70cc0b5560 2024-04-26T21:47:25.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.608+0000 7f70c2a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 0x7f70a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.608+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f70b408af20 con 0x7f70cc0b5560 2024-04-26T21:47:25.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.608+0000 7f70caa00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 0x7f70a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.609+0000 7f70caa00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 0x7f70a806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006a90 tx=0x7f70bc006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.612+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70b40593e0 con 0x7f70cc0b5560 2024-04-26T21:47:25.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.884+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f70cc0b0b00 con 0x7f70cc0b5560 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.890+0000 7f70c2a00640 1 -- 172.21.15.106:0/3523685145 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f70b4058ce0 con 0x7f70cc0b5560 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q== 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-26T21:47:25.890 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:47:25.891 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-26T21:47:25.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 msgr2=0x7f70a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 0x7f70a806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006a90 tx=0x7f70bc006a20 comp rx=0 tx=0).stop 2024-04-26T21:47:25.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 msgr2=0x7f70cc09e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c400 tx=0x7f70b400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 shutdown_connections 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70cc15fe20 0x7f70cc09e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70cc0b5560 0x7f70cc09e0c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f70a806c8f0 0x7f70a806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 --2- 172.21.15.106:0/3523685145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70cc06f450 0x7f70cc15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 >> 172.21.15.106:0/3523685145 conn(0x7f70cc0b2a30 msgr2=0x7f70cc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 shutdown_connections 2024-04-26T21:47:25.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:25.893+0000 7f70d1a00640 1 -- 172.21.15.106:0/3523685145 wait complete. 2024-04-26T21:47:25.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.106+0000 7fda16e00640 1 Processor -- start 2024-04-26T21:47:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.106+0000 7fda16e00640 1 -- start start 2024-04-26T21:47:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda10146680 0x7fda100b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda100d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 0x7fda10160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fda1006e970 con 0x7fda10146680 2024-04-26T21:47:26.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fda100d17f0 con 0x7fda10160930 2024-04-26T21:47:26.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fda10146a60 con 0x7fda100d0e70 2024-04-26T21:47:26.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.108+0000 7fda15a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda10146680 0x7fda100b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda10146680 0x7fda100b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53608/0 (socket says 172.21.15.106:53608) 2024-04-26T21:47:26.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 -- 172.21.15.106:0/3016259444 learned_addr learned my addr 172.21.15.106:0/3016259444 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda16400640 1 --2- 172.21.15.106:0/3016259444 >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 0x7fda10160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 -- 172.21.15.106:0/3016259444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 msgr2=0x7fda100d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15000640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda100d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda100d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 -- 172.21.15.106:0/3016259444 >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 msgr2=0x7fda10160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 --2- 172.21.15.106:0/3016259444 >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 0x7fda10160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda15a00640 1 -- 172.21.15.106:0/3016259444 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1006dd60 con 0x7fda10146680 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.109+0000 7fda16400640 1 --2- 172.21.15.106:0/3016259444 >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 0x7fda10160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.110+0000 7fda15a00640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10146680 0x7fda100b5990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda100d1ad0 tx=0x7fda0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00daa9e02458129 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.110+0000 7fda0fe00640 1 -- 172.21.15.106:0/3016259444 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04037070 con 0x7fda10146680 2024-04-26T21:47:26.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.110+0000 7fda0fe00640 1 -- 172.21.15.106:0/3016259444 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda04032440 con 0x7fda10146680 2024-04-26T21:47:26.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.110+0000 7fda0fe00640 1 -- 172.21.15.106:0/3016259444 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04032810 con 0x7fda10146680 2024-04-26T21:47:26.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 -- 172.21.15.106:0/3016259444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10146680 msgr2=0x7fda100b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10146680 0x7fda100b5990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda100d1ad0 tx=0x7fda0402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 -- 172.21.15.106:0/3016259444 shutdown_connections 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 --2- 172.21.15.106:0/3016259444 >> v2:172.21.15.204:3300/0 conn(0x7fda10160930 0x7fda10160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda100d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 --2- 172.21.15.106:0/3016259444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10146680 0x7fda100b5990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 -- 172.21.15.106:0/3016259444 >> 172.21.15.106:0/3016259444 conn(0x7fda1006c740 msgr2=0x7fda1006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 -- 172.21.15.106:0/3016259444 shutdown_connections 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.111+0000 7fda16e00640 1 -- 172.21.15.106:0/3016259444 wait complete. 2024-04-26T21:47:26.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.113+0000 7fda16e00640 1 Processor -- start 2024-04-26T21:47:26.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.113+0000 7fda16e00640 1 -- start start 2024-04-26T21:47:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda101f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda10146680 0x7fda1015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda1006e600 con 0x7fda10160930 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda1006e320 con 0x7fda10146680 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fda100cd6a0 con 0x7fda100d0e70 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53618/0 (socket says 172.21.15.106:53618) 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda16400640 1 -- 172.21.15.106:0/1590460353 learned_addr learned my addr 172.21.15.106:0/1590460353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.114+0000 7fda15000640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda10146680 0x7fda1015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 -- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 msgr2=0x7fda101f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda101f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 -- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda10146680 msgr2=0x7fda1015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda10146680 0x7fda1015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 -- 172.21.15.106:0/1590460353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda101453d0 con 0x7fda10160930 2024-04-26T21:47:26.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16400640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda0800cd50 tx=0x7fda0800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800f680 con 0x7fda10160930 2024-04-26T21:47:26.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0800f840 con 0x7fda10160930 2024-04-26T21:47:26.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda101437e0 con 0x7fda10160930 2024-04-26T21:47:26.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800fc30 con 0x7fda10160930 2024-04-26T21:47:26.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.115+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda10145950 con 0x7fda10160930 2024-04-26T21:47:26.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.116+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda100693c0 con 0x7fda10160930 2024-04-26T21:47:26.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.117+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda080213f0 con 0x7fda10160930 2024-04-26T21:47:26.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.118+0000 7fda0d600640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.118+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fda0808c910 con 0x7fda10160930 2024-04-26T21:47:26.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.118+0000 7fda15a00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.119+0000 7fda15a00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 0x7fd9f006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda04034000 tx=0x7fda0402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.123+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda08014030 con 0x7fda10160930 2024-04-26T21:47:26.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.343+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda10073df0 con 0x7fda10160930 2024-04-26T21:47:26.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.344+0000 7fda0d600640 1 -- 172.21.15.106:0/1590460353 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fda0805ad30 con 0x7fda10160930 2024-04-26T21:47:26.344 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:26.344 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:26.344 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-26T21:47:26.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 msgr2=0x7fd9f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 0x7fd9f006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda04034000 tx=0x7fda0402c000 comp rx=0 tx=0).stop 2024-04-26T21:47:26.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 msgr2=0x7fda100738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda0800cd50 tx=0x7fda0800a650 comp rx=0 tx=0).stop 2024-04-26T21:47:26.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 shutdown_connections 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda10160930 0x7fda100738b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda10146680 0x7fda1015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 --2- 172.21.15.106:0/1590460353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda100d0e70 0x7fda101f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 >> 172.21.15.106:0/1590460353 conn(0x7fda1006c740 msgr2=0x7fda1015e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 shutdown_connections 2024-04-26T21:47:26.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.346+0000 7fda16e00640 1 -- 172.21.15.106:0/1590460353 wait complete. 2024-04-26T21:47:26.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:47:14.908611+0000', 'modified': '2024-04-26T21:47:17.987975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34541': {'gid': 34541, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.204:6835/107496404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 107496404}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 107496404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34541, 'qdb_cluster': [34541]} 2024-04-26T21:47:26.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:47:26.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:47:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.980+0000 7f13fbe00640 1 Processor -- start 2024-04-26T21:47:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.981+0000 7f13fbe00640 1 -- start start 2024-04-26T21:47:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13fbe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13fc0d5990 0x7f13fc163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0b82b0 0x7f13fc0b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13fbe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13fc0d3f50 con 0x7f13fc15f1b0 2024-04-26T21:47:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13fbe00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f13fc0d36b0 con 0x7f13fc0d5990 2024-04-26T21:47:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13fbe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc0d3990 con 0x7f13fc0b82b0 2024-04-26T21:47:26.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13faa00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f13fc15f1b0 0x7f13fc162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43210/0 (socket says 172.21.15.106:43210) 2024-04-26T21:47:26.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.982+0000 7f13faa00640 1 -- 172.21.15.106:0/3276923520 learned_addr learned my addr 172.21.15.106:0/3276923520 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fa000640 1 --2- 172.21.15.106:0/3276923520 >> v2:172.21.15.106:3300/0 conn(0x7f13fc0d5990 0x7f13fc163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2736815701 0 0) 0x7f13fc0d3f50 con 0x7f13fc15f1b0 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13dc003520 con 0x7f13fc15f1b0 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fb400640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0b82b0 0x7f13fc0b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fa000640 1 -- 172.21.15.106:0/3276923520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0b82b0 msgr2=0x7f13fc0b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fa000640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0b82b0 0x7f13fc0b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4081243362 0 0) 0x7f13dc003520 con 0x7f13fc15f1b0 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fa000640 1 -- 172.21.15.106:0/3276923520 >> v1:172.21.15.106:6789/0 conn(0x7f13fc15f1b0 legacy=0x7f13fc162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.983+0000 7f13fa000640 1 -- 172.21.15.106:0/3276923520 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc0d3c70 con 0x7f13fc0d5990 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.984+0000 7f13fa000640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc0d5990 0x7f13fc163210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009d90 tx=0x7f13e402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23335293f018008a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec001670 con 0x7f13fc0d5990 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e4007540 con 0x7f13fc0d5990 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13f9600640 1 -- 172.21.15.106:0/3276923520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4007910 con 0x7f13fc0d5990 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 -- 172.21.15.106:0/3276923520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc0d5990 msgr2=0x7f13fc163210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc0d5990 0x7f13fc163210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009d90 tx=0x7f13e402a5f0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 -- 172.21.15.106:0/3276923520 shutdown_connections 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0b82b0 0x7f13fc0b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3276923520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc0d5990 0x7f13fc163210 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 -- 172.21.15.106:0/3276923520 >> 172.21.15.106:0/3276923520 conn(0x7f13fc09e230 msgr2=0x7f13fc0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:26.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.985+0000 7f13fbe00640 1 -- 172.21.15.106:0/3276923520 shutdown_connections 2024-04-26T21:47:26.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.986+0000 7f13fbe00640 1 -- 172.21.15.106:0/3276923520 wait complete. 2024-04-26T21:47:26.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.987+0000 7f13fbe00640 1 Processor -- start 2024-04-26T21:47:26.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.987+0000 7f13fbe00640 1 -- start start 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 0x7f13fc15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0d5990 0x7f13fc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc163750 con 0x7f13fc15f1b0 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc0d33a0 con 0x7f13fc0b82b0 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fbe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc1638d0 con 0x7f13fc0d5990 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.988+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53624/0 (socket says 172.21.15.106:53624) 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 -- 172.21.15.106:0/3922215386 learned_addr learned my addr 172.21.15.106:0/3922215386 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:26.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13faa00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 0x7f13fc15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 -- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0d5990 msgr2=0x7f13fc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0d5990 0x7f13fc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 -- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 msgr2=0x7f13fc15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 0x7f13fc15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 -- 172.21.15.106:0/3922215386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc0cd8e0 con 0x7f13fc15f1b0 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13faa00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 0x7f13fc15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:26.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13fb400640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d830 tx=0x7f13f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13f2a00640 1 -- 172.21.15.106:0/3922215386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f400b6c0 con 0x7f13fc15f1b0 2024-04-26T21:47:26.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.989+0000 7f13f2a00640 1 -- 172.21.15.106:0/3922215386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4019070 con 0x7f13fc15f1b0 2024-04-26T21:47:26.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.990+0000 7f13f2a00640 1 -- 172.21.15.106:0/3922215386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f400bac0 con 0x7f13fc15f1b0 2024-04-26T21:47:26.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.990+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc0cdbc0 con 0x7f13fc15f1b0 2024-04-26T21:47:26.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.990+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc0ccb90 con 0x7f13fc15f1b0 2024-04-26T21:47:26.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.992+0000 7f13f2a00640 1 -- 172.21.15.106:0/3922215386 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13f401e460 con 0x7f13fc15f1b0 2024-04-26T21:47:26.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.993+0000 7f13f2a00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 0x7f13d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.993+0000 7f13f2a00640 1 -- 172.21.15.106:0/3922215386 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f13f408c200 con 0x7f13fc15f1b0 2024-04-26T21:47:26.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.993+0000 7f13faa00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 0x7f13d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.993+0000 7f1404589740 1 Processor -- start 2024-04-26T21:47:26.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.994+0000 7f1404589740 1 -- start start 2024-04-26T21:47:26.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.994+0000 7f13faa00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 0x7f13d806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001eb0 tx=0x7f13ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13068b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1305ed600 con 0x55c1306c5e80 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c13058a100 con 0x55c1306897b0 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c1305e95c0 con 0x55c1306b4270 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53634/0 (socket says 172.21.15.106:53634) 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 -- 172.21.15.106:0/240849567 learned_addr learned my addr 172.21.15.106:0/240849567 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:26.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 -- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 msgr2=0x55c1306b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 -- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 msgr2=0x55c13068b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13faa00640 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13068b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13068b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.995+0000 7f13fa000640 1 -- 172.21.15.106:0/240849567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1302ff2d0 con 0x55c1306c5e80 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.996+0000 7f13fa000640 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009d90 tx=0x7f13e402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99b15a162e8a059 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.996+0000 7f13eb000640 1 -- 172.21.15.106:0/240849567 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4040070 con 0x55c1306c5e80 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.996+0000 7f13eb000640 1 -- 172.21.15.106:0/240849567 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e4007740 con 0x55c1306c5e80 2024-04-26T21:47:26.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.996+0000 7f13eb000640 1 -- 172.21.15.106:0/240849567 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4007b10 con 0x55c1306c5e80 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 -- 172.21.15.106:0/240849567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 msgr2=0x55c130689b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009d90 tx=0x7f13e402c000 comp rx=0 tx=0).stop 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 -- 172.21.15.106:0/240849567 shutdown_connections 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c130689b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 --2- 172.21.15.106:0/240849567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13068b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 -- 172.21.15.106:0/240849567 >> 172.21.15.106:0/240849567 conn(0x55c13061dd10 msgr2=0x55c1306cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 -- 172.21.15.106:0/240849567 shutdown_connections 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.997+0000 7f1404589740 1 -- 172.21.15.106:0/240849567 wait complete. 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.998+0000 7f1404589740 1 -- start start 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.998+0000 7f1404589740 1 Processor -- start 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306897b0 0x55c1306c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 0x55c1306c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c130574f70 con 0x55c1306c5e80 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c1306b1dc0 con 0x55c1306b4270 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c1306b7710 con 0x55c1306897b0 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f1404589740 10 client.0 _ll_register_callbacks cb 0x55c1305e3480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53642/0 (socket says 172.21.15.106:53642) 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f13fb400640 1 -- 172.21.15.106:0/1672183189 learned_addr learned my addr 172.21.15.106:0/1672183189 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:26.999+0000 7f13fa000640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 0x55c1306c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 -- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306897b0 msgr2=0x55c1306c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306897b0 0x55c1306c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 -- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 msgr2=0x55c1306c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 0x55c1306c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 -- 172.21.15.106:0/1672183189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1303c96b0 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fa000640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 0x55c1306c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13fb400640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f13f40562f0 tx=0x7f13f4059e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51bff2b6dae7029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13e3400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4072020 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f13f40721c0 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e4007740 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f13f4077050 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f1404589740 10 client.5463 fetch_fsmap learned FSMap version 16 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.000+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55c1306b7450 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e4007740 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f13f4013070 con 0x55c1306c5e80 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 10 client.5463 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 10 client.5463 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:47:27.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 20 client.5463 populate_metadata read hostname 'smithi106' 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b4870 0x55c1306b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 4 client.5463 mds_command: new command op to 34541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 --> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1305fb210 con 0x55c1306b4870 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.001+0000 7f13faa00640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b4870 0x55c1306b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.002+0000 7f13faa00640 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b4870 0x55c1306b66c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f7e15f45fc64f0 server_cookie=8ca92b733a24c411 in_seq=0 out_seq=0 2024-04-26T21:47:27.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.002+0000 7f13e3400640 10 client.5463 ms_handle_connect on v2:172.21.15.204:6833/107496404 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f13e3400640 1 -- 172.21.15.106:0/1672183189 <== mds.0 v2:172.21.15.204:6833/107496404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1305fb210 con 0x55c1306b4870 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f13e3400640 10 client.5463 handle_command_reply: tid=0 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 client.5463 shutdown 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 msgr2=0x55c1306c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f13f40562f0 tx=0x7f13f4059e90 comp rx=0 tx=0).stop 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 shutdown_connections 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b4870 0x55c1306b66c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306c8920 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306b4270 0x55c1306c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 --2- 172.21.15.106:0/1672183189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306897b0 0x55c1306c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 >> 172.21.15.106:0/1672183189 conn(0x55c13061dd10 msgr2=0x55c1306d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f13e3400640 0 client.5463 ms_handle_reset on v2:172.21.15.204:6833/107496404 2024-04-26T21:47:27.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 shutdown_connections 2024-04-26T21:47:27.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.005+0000 7f1404589740 1 -- 172.21.15.106:0/1672183189 wait complete. 2024-04-26T21:47:27.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.006+0000 7f1404589740 20 client.5463 trim_cache size 0 max 16384 2024-04-26T21:47:27.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.025+0000 7f1404589740 1 Processor -- start 2024-04-26T21:47:27.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.025+0000 7f1404589740 1 -- start start 2024-04-26T21:47:27.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13067fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc200350 con 0x55c1306c5e80 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1ff790 con 0x55c1306897b0 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.026+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc1ffcc0 con 0x55c1306b4270 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53654/0 (socket says 172.21.15.106:53654) 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 -- 172.21.15.106:0/3682910043 learned_addr learned my addr 172.21.15.106:0/3682910043 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13faa00640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13067fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fa000640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 -- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 msgr2=0x55c1306cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 -- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 msgr2=0x55c13067fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13067fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 -- 172.21.15.106:0/3682910043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c130687b10 con 0x55c1306c5e80 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13faa00640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13067fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:27.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fa000640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:27.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.027+0000 7f13fb400640 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f13f4059b90 tx=0x7f13f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfb3440cc5dc08a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f13ea600640 1 -- 172.21.15.106:0/3682910043 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4072040 con 0x55c1306c5e80 2024-04-26T21:47:27.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f13ea600640 1 -- 172.21.15.106:0/3682910043 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4072460 con 0x55c1306c5e80 2024-04-26T21:47:27.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f13ea600640 1 -- 172.21.15.106:0/3682910043 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4062a70 con 0x55c1306c5e80 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 -- 172.21.15.106:0/3682910043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 msgr2=0x55c1306de190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f13f4059b90 tx=0x7f13f4020000 comp rx=0 tx=0).stop 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 -- 172.21.15.106:0/3682910043 shutdown_connections 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 --2- 172.21.15.106:0/3682910043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c13067fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 -- 172.21.15.106:0/3682910043 >> 172.21.15.106:0/3682910043 conn(0x55c13061dd10 msgr2=0x55c1306b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.028+0000 7f1404589740 1 -- 172.21.15.106:0/3682910043 shutdown_connections 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.029+0000 7f1404589740 1 -- 172.21.15.106:0/3682910043 wait complete. 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.029+0000 7f1404589740 1 -- start start 2024-04-26T21:47:27.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.029+0000 7f1404589740 1 Processor -- start 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c1306df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1306b46f0 con 0x55c1306c5e80 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c1306df200 con 0x55c1306897b0 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.030+0000 7f1404589740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c1306b4d40 con 0x55c1306b4270 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f1404589740 10 client.0 _ll_register_callbacks cb 0x7f13f4056190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53656/0 (socket says 172.21.15.106:53656) 2024-04-26T21:47:27.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 -- 172.21.15.106:0/2983977309 learned_addr learned my addr 172.21.15.106:0/2983977309 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 -- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 msgr2=0x55c1306de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 -- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 msgr2=0x55c1306df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c1306df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 -- 172.21.15.106:0/2983977309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c130688750 con 0x55c1306c5e80 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13fb400640 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d800 tx=0x7f13f4006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a603b86c1b72d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.031+0000 7f13e3400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4072040 con 0x55c1306c5e80 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f13f4056500 con 0x55c1306c5e80 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f4072460 con 0x55c1306c5e80 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f13f4062a70 con 0x55c1306c5e80 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 10 client.5469 fetch_fsmap learned FSMap version 16 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55c1307525d0 con 0x55c1306c5e80 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4072460 con 0x55c1306c5e80 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f13f40592e0 con 0x55c1306c5e80 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 10 client.5469 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 10 client.5469 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 20 client.5469 populate_metadata read hostname 'smithi106' 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b2770 0x55c130751550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 4 client.5469 mds_command: new command op to 34541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.032+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 --> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] -- command(tid 0: {"prefix": "status"}) -- 0x55c1305f5080 con 0x55c1306b2770 2024-04-26T21:47:27.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.033+0000 7f13faa00640 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b2770 0x55c130751550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f13faa00640 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b2770 0x55c130751550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb631dd1e2ece8c server_cookie=5097add769cee00f in_seq=0 out_seq=0 2024-04-26T21:47:27.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f13e3400640 10 client.5469 ms_handle_connect on v2:172.21.15.204:6833/107496404 2024-04-26T21:47:27.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f13e3400640 1 -- 172.21.15.106:0/2983977309 <== mds.0 v2:172.21.15.204:6833/107496404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c1305f5080 con 0x55c1306b2770 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f13e3400640 10 client.5469 handle_command_reply: tid=0 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 client.5469 shutdown 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 msgr2=0x55c1306de6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d800 tx=0x7f13f4006d70 comp rx=0 tx=0).stop 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 shutdown_connections 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:6833/107496404,v1:172.21.15.204:6835/107496404] conn(0x55c1306b2770 0x55c130751550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1306c5e80 0x55c1306de6d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.034+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c1306b4270 0x55c1306de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f13e3400640 0 client.5469 ms_handle_reset on v2:172.21.15.204:6833/107496404 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f1404589740 1 --2- 172.21.15.106:0/2983977309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1306897b0 0x55c1306df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 >> 172.21.15.106:0/2983977309 conn(0x55c13061dd10 msgr2=0x55c1306c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 shutdown_connections 2024-04-26T21:47:27.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f1404589740 1 -- 172.21.15.106:0/2983977309 wait complete. 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f1404589740 20 client.5469 trim_cache size 0 max 16384 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e6ee84ed-cbe5-4ff1-aa0d-97a291446523", 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 34541, 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-26T21:47:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 11.544284340000001, 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 16, 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 33, 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 33, 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 20.391737551999999 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:47:27.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 msgr2=0x7f13d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 0x7f13d806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001eb0 tx=0x7f13ec009040 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 msgr2=0x7f13fc0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.035+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d830 tx=0x7f13f400dcf0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 shutdown_connections 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13fc15f1b0 0x7f13fc0ce970 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13d806c920 0x7f13d806ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13fc0d5990 0x7f13fc0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 --2- 172.21.15.106:0/3922215386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13fc0b82b0 0x7f13fc15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 >> 172.21.15.106:0/3922215386 conn(0x7f13fc09e230 msgr2=0x7f13fc0b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 shutdown_connections 2024-04-26T21:47:27.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.036+0000 7f13fbe00640 1 -- 172.21.15.106:0/3922215386 wait complete. 2024-04-26T21:47:27.052 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:47:27.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.250+0000 7eff42400640 1 Processor -- start 2024-04-26T21:47:27.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.250+0000 7eff42400640 1 -- start start 2024-04-26T21:47:27.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff42400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff42400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7eff3c0ca540 0x7eff3c168b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff42400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3c06e680 con 0x7eff3c0cc240 2024-04-26T21:47:27.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff42400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7eff3c0cc6a0 con 0x7eff3c0ca540 2024-04-26T21:47:27.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff42400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c070130 con 0x7eff3c147d50 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff41000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.252+0000 7eff41000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:36254/0 (socket says 172.21.15.106:36254) 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41000640 1 -- 172.21.15.106:0/3995360301 learned_addr learned my addr 172.21.15.106:0/3995360301 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41a00640 1 --2- 172.21.15.106:0/3995360301 >> v2:172.21.15.204:3300/0 conn(0x7eff3c0ca540 0x7eff3c168b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41000640 1 -- 172.21.15.106:0/3995360301 >> v2:172.21.15.204:3300/0 conn(0x7eff3c0ca540 msgr2=0x7eff3c168b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41000640 1 --2- 172.21.15.106:0/3995360301 >> v2:172.21.15.204:3300/0 conn(0x7eff3c0ca540 0x7eff3c168b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41000640 1 -- 172.21.15.106:0/3995360301 >> v1:172.21.15.204:6789/0 conn(0x7eff3c0cc240 legacy=0x7eff3c0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.253+0000 7eff41000640 1 -- 172.21.15.106:0/3995360301 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c0caa40 con 0x7eff3c147d50 2024-04-26T21:47:27.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.255+0000 7eff41000640 1 --2- 172.21.15.106:0/3995360301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff30009870 tx=0x7eff3002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930eccb5150a2db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff3b400640 1 -- 172.21.15.106:0/3995360301 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30037070 con 0x7eff3c147d50 2024-04-26T21:47:27.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff3b400640 1 -- 172.21.15.106:0/3995360301 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff30032440 con 0x7eff3c147d50 2024-04-26T21:47:27.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff3b400640 1 -- 172.21.15.106:0/3995360301 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30032810 con 0x7eff3c147d50 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 -- 172.21.15.106:0/3995360301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 msgr2=0x7eff3c0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 --2- 172.21.15.106:0/3995360301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff30009870 tx=0x7eff3002a270 comp rx=0 tx=0).stop 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 -- 172.21.15.106:0/3995360301 shutdown_connections 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 --2- 172.21.15.106:0/3995360301 >> v2:172.21.15.204:3300/0 conn(0x7eff3c0ca540 0x7eff3c168b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 --2- 172.21.15.106:0/3995360301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c147d50 0x7eff3c0cbd00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.256+0000 7eff42400640 1 -- 172.21.15.106:0/3995360301 >> 172.21.15.106:0/3995360301 conn(0x7eff3c0d0ae0 msgr2=0x7eff3c0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.257+0000 7eff42400640 1 -- 172.21.15.106:0/3995360301 shutdown_connections 2024-04-26T21:47:27.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.257+0000 7eff42400640 1 -- 172.21.15.106:0/3995360301 wait complete. 2024-04-26T21:47:27.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.259+0000 7eff42400640 1 Processor -- start 2024-04-26T21:47:27.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.259+0000 7eff42400640 1 -- start start 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c0ca540 0x7eff3c0acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 0x7eff3c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c06ec00 con 0x7eff3c147d50 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c06e920 con 0x7eff3c0cc240 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.260+0000 7eff42400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c0ce6b0 con 0x7eff3c0ca540 2024-04-26T21:47:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53672/0 (socket says 172.21.15.106:53672) 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 -- 172.21.15.106:0/2696238681 learned_addr learned my addr 172.21.15.106:0/2696238681 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41000640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c0ca540 0x7eff3c0acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff3be00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 0x7eff3c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 -- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c0ca540 msgr2=0x7eff3c0acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c0ca540 0x7eff3c0acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 -- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 msgr2=0x7eff3c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 0x7eff3c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff41a00640 1 -- 172.21.15.106:0/2696238681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c0afff0 con 0x7eff3c147d50 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.261+0000 7eff3be00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 0x7eff3c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff41a00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ef90 tx=0x7eff3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34018020 con 0x7eff3c147d50 2024-04-26T21:47:27.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff34014480 con 0x7eff3c147d50 2024-04-26T21:47:27.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c0b3d60 con 0x7eff3c147d50 2024-04-26T21:47:27.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34014870 con 0x7eff3c147d50 2024-04-26T21:47:27.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.262+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c0b02d0 con 0x7eff3c147d50 2024-04-26T21:47:27.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.263+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c0693c0 con 0x7eff3c147d50 2024-04-26T21:47:27.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.264+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff34005d10 con 0x7eff3c147d50 2024-04-26T21:47:27.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.265+0000 7eff38c00640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.266+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7eff3408ca40 con 0x7eff3c147d50 2024-04-26T21:47:27.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.266+0000 7eff41000640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.267+0000 7eff41000640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 0x7eff1c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff30034000 tx=0x7eff3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.270+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff34022030 con 0x7eff3c147d50 2024-04-26T21:47:27.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.510+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7eff3c147570 con 0x7eff3c147d50 2024-04-26T21:47:27.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.511+0000 7eff38c00640 1 -- 172.21.15.106:0/2696238681 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7eff34010040 con 0x7eff3c147d50 2024-04-26T21:47:27.511 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 msgr2=0x7eff1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 0x7eff1c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff30034000 tx=0x7eff3002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 msgr2=0x7eff3c0abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ef90 tx=0x7eff3400c490 comp rx=0 tx=0).stop 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 shutdown_connections 2024-04-26T21:47:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff3c147d50 0x7eff3c0abbd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3c0cc240 0x7eff3c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 --2- 172.21.15.106:0/2696238681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff3c0ca540 0x7eff3c0acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 >> 172.21.15.106:0/2696238681 conn(0x7eff3c0d0ae0 msgr2=0x7eff3c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 shutdown_connections 2024-04-26T21:47:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27.514+0000 7eff42400640 1 -- 172.21.15.106:0/2696238681 wait complete. 2024-04-26T21:47:27.532 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:47:27.533 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:27.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.533 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-26T21:47:27.584 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:47:27.584 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:47:27.584 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:47:27.584 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.584 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-04-26T21:47:27.585 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c49c:8bff:fe17:2a52/64 scope link 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::480c:21ff:fe55:3af6/64 scope link 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 62:45:ae:c9:5b:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:27.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::6045:aeff:fec9:5b20/64 scope link 2024-04-26T21:47:27.587 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:27.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.588 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:47:27.663 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-26T21:47:27.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.668 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-26T21:47:27.722 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-26T21:47:27.722 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:47:27.722 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:47:27.722 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:47:27.722 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:47:27.724 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:47:27.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:47:27.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:27.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.935 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:47:27.982 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-26T21:47:28.112 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T21:47:28.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:28.112 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:28.165 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:28.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:28.165 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:28.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:47:28.342 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:47:28.398 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:47:28.399 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:28.400 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:28.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:28.401 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:47:28.448 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:47:28.448 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T21:47:28.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:47:28.523 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:47:28.565 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:47:28.566 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:47:28.567 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:28.568 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:28.569 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:47:28.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-26T21:47:28.621+0000 7f8a9fb94480 -1 init, newargv = 0x55fd649982d0 newargc=15 2024-04-26T21:47:28.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[47428]: starting ceph client 2024-04-26T21:47:28.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[47428]: starting fuse 2024-04-26T21:47:28.651 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-26T21:47:28.652 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:47:28.652 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:47:28.653 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-26T21:47:28.654 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-26T21:47:28.843 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.47428.asok 2024-04-26T21:47:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:28.860 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47428.asok 2024-04-26T21:47:28.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:28.860 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47428.asok status 2024-04-26T21:47:29.080 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:47:29.080 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "47428", 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5478, 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-26T21:47:29.081 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5478 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 2189758624 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:47:29.082 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 2189758624 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.5478 192.168.0.1:0/2189758624", 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/2189758624", 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 16, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 33, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-26T21:47:29.083 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:47:29.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.094 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-26T21:47:29.095 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:29.146 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-26T21:47:29.146 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:29.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.147 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:29.223 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:47:29.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.424+0000 7f1f49000640 1 Processor -- start 2024-04-26T21:47:29.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.424+0000 7f1f49000640 1 -- start start 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f49000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f4406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f49000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f44073f40 con 0x7f1f4415cb70 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f49000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4406d090 con 0x7f1f4415cfd0 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f49000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f440af460 con 0x7f1f4406c770 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f41400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1f4415cb70 0x7f1f44073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54302/0 (socket says 172.21.15.106:54302) 2024-04-26T21:47:29.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f41400640 1 -- 172.21.15.106:0/3373241622 learned_addr learned my addr 172.21.15.106:0/3373241622 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f41e00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f4406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574034055 0 0) 0x7f1f44073f40 con 0x7f1f4415cb70 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.426+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f28003520 con 0x7f1f4415cb70 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1279946572 0 0) 0x7f1f28003520 con 0x7f1f4415cb70 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 msgr2=0x7f1f4406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f4406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 >> v1:172.21.15.204:6789/0 conn(0x7f1f4415cfd0 legacy=0x7f1f44073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f440ae850 con 0x7f1f4415cb70 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f1f30001670 con 0x7f1f4415cb70 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 >> v1:172.21.15.106:6789/0 conn(0x7f1f4415cb70 legacy=0x7f1f44073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f440af460 con 0x7f1f28003f60 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800baa0 con 0x7f1f28004d40 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.427+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2800deb0 con 0x7f1f4406c770 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41e00640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f42800640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 -- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 msgr2=0x7f1f280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4406c770 0x7f1f280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 -- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f28004d40 msgr2=0x7f1f280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f41400640 1 -- 172.21.15.106:0/3373241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f28003520 con 0x7f1f28003f60 2024-04-26T21:47:29.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.428+0000 7f1f42800640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.429+0000 7f1f41400640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f28004800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c210 tx=0x7f1f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0a93517a731083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.429+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30007d30 con 0x7f1f28003f60 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.429+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f30005af0 con 0x7f1f28003f60 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.429+0000 7f1f40a00640 1 -- 172.21.15.106:0/3373241622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30007070 con 0x7f1f28003f60 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 -- 172.21.15.106:0/3373241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 msgr2=0x7f1f28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 --2- 172.21.15.106:0/3373241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f28004800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c210 tx=0x7f1f28005830 comp rx=0 tx=0).stop 2024-04-26T21:47:29.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f42800640 1 -- 172.21.15.106:0/3373241622 reap_dead start 2024-04-26T21:47:29.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 -- 172.21.15.106:0/3373241622 shutdown_connections 2024-04-26T21:47:29.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 -- 172.21.15.106:0/3373241622 >> 172.21.15.106:0/3373241622 conn(0x7f1f440b2d50 msgr2=0x7f1f440b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 -- 172.21.15.106:0/3373241622 shutdown_connections 2024-04-26T21:47:29.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.430+0000 7f1f49000640 1 -- 172.21.15.106:0/3373241622 wait complete. 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.432+0000 7f1f49000640 1 Processor -- start 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.432+0000 7f1f49000640 1 -- start start 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 0x7f1f4406f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4409cb90 0x7f1f4406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f440af0f0 con 0x7f1f28003f60 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f440aee10 con 0x7f1f4409c710 2024-04-26T21:47:29.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f49000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4415c600 con 0x7f1f4409cb90 2024-04-26T21:47:29.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f41e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f41e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54982/0 (socket says 172.21.15.106:54982) 2024-04-26T21:47:29.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.433+0000 7f1f41e00640 1 -- 172.21.15.106:0/1311221728 learned_addr learned my addr 172.21.15.106:0/1311221728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41400640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 0x7f1f4406f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 -- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4409cb90 msgr2=0x7f1f4406fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4409cb90 0x7f1f4406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 -- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 msgr2=0x7f1f4406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 0x7f1f4406f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 -- 172.21.15.106:0/1311221728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441f4530 con 0x7f1f28003f60 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41400640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 0x7f1f4406f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f41e00640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1f340119a0 tx=0x7f1f34011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400faf0 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3400fc90 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34014440 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f440b1590 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.434+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f44072c30 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.436+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4415bdc0 con 0x7f1f28003f60 2024-04-26T21:47:29.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.437+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f34012040 con 0x7f1f28003f60 2024-04-26T21:47:29.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.438+0000 7f1f3a000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 0x7f1f2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.438+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1f3408c250 con 0x7f1f28003f60 2024-04-26T21:47:29.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.438+0000 7f1f41400640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 0x7f1f2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.439+0000 7f1f41400640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 0x7f1f2406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f30033000 tx=0x7f1f3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.443+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f4415bdc0 con 0x7f1f28003f60 2024-04-26T21:47:29.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.687+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f440cd430 con 0x7f1f28003f60 2024-04-26T21:47:29.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.687+0000 7f1f3a000640 1 -- 172.21.15.106:0/1311221728 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f1f34019030 con 0x7f1f28003f60 2024-04-26T21:47:29.687 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 msgr2=0x7f1f2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 0x7f1f2406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f30033000 tx=0x7f1f3002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 msgr2=0x7f1f441f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1f340119a0 tx=0x7f1f34011e60 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 shutdown_connections 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f2406c8f0 0x7f1f2406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f4409cb90 0x7f1f4406fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4409c710 0x7f1f4406f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 --2- 172.21.15.106:0/1311221728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f28003f60 0x7f1f441f3ff0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 >> 172.21.15.106:0/1311221728 conn(0x7f1f440b2d50 msgr2=0x7f1f44158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.690+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 shutdown_connections 2024-04-26T21:47:29.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:29.691+0000 7f1f49000640 1 -- 172.21.15.106:0/1311221728 wait complete. 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:47:29.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:47:29.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.710 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:29.769 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42529sec preferred_lft 42529sec 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:3a:e4:9f:22:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::2818:afff:fe5c:ca4c/64 scope link 2024-04-26T21:47:29.770 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ce:52:1d:14:cf:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::cc52:1dff:fe14:cf6d/64 scope link 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:3a:e4:9f:22:6a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::6c3a:e4ff:fe9f:226a/64 scope link 2024-04-26T21:47:29.771 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:29.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.772 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:47:29.852 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-26T21:47:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.856 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-04-26T21:47:29.905 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-04-26T21:47:29.905 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:47:29.905 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:47:29.906 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:47:29.906 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:47:29.907 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:47:30.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:47:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:47:30.106 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:47:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:30.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:30.115 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:47:30.165 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:47:30.165 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:47:30.166 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:47:30.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:30.167 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-04-26T21:47:30.248 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T21:47:30.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:30.248 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:30.301 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:47:30.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:30.301 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:30.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:47:30.418 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:30.473 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:47:30.524 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:47:30.524 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T21:47:30.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:47:30.597 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:47:30.640 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:30.641 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:30.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:30.642 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:47:30.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-04-26T21:47:30.700+0000 7f847cd20480 -1 init, newargv = 0x557e2bf5e000 newargc=15 2024-04-26T21:47:30.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[40500]: starting ceph client 2024-04-26T21:47:30.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[40500]: starting fuse 2024-04-26T21:47:30.725 INFO:teuthology.orchestra.run.smithi204.stdout:41 2024-04-26T21:47:30.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T21:47:30.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:47:30.725 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-04-26T21:47:30.725 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-04-26T21:47:30.726 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-04-26T21:47:30.727 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-04-26T21:47:30.914 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.40500.asok 2024-04-26T21:47:30.919 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:47:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:30.931 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40500.asok 2024-04-26T21:47:30.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:30.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40500.asok status 2024-04-26T21:47:31.155 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "40500", 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 34610, 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-04-26T21:47:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 34610 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 924789180 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 924789180 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:47:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.34610 192.168.0.1:0/924789180", 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/924789180", 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 16, 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 33, 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-04-26T21:47:31.158 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:47:31.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:31.170 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-04-26T21:47:31.170 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:31.222 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-04-26T21:47:31.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:31.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:31.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:31.303 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.523+0000 7fa86de00640 1 Processor -- start 2024-04-26T21:47:31.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.523+0000 7fa86de00640 1 -- start start 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa86de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa868163720 0x7fa86806f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa86de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa86806e430 0x7fa8680b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa86de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8680b4b90 con 0x7fa8680cd6a0 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa86de00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa86806e000 con 0x7fa868163720 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa86de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680b5760 con 0x7fa86806e430 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa868163720 0x7fa86806f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa868163720 0x7fa86806f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54984/0 (socket says 172.21.15.106:54984) 2024-04-26T21:47:31.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 -- 172.21.15.106:0/3930714563 learned_addr learned my addr 172.21.15.106:0/3930714563 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:31.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 -- 172.21.15.106:0/3930714563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa86806e430 msgr2=0x7fa8680b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:31.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 --2- 172.21.15.106:0/3930714563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa86806e430 0x7fa8680b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 -- 172.21.15.106:0/3930714563 >> v1:172.21.15.204:6789/0 conn(0x7fa8680cd6a0 legacy=0x7fa868071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.525+0000 7fa866a00640 1 -- 172.21.15.106:0/3930714563 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680b4e30 con 0x7fa868163720 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.527+0000 7fa866a00640 1 --2- 172.21.15.106:0/3930714563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa86806f390 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa858009870 tx=0x7fa85802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6270cb962d00647 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.527+0000 7fa866000640 1 -- 172.21.15.106:0/3930714563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858037070 con 0x7fa868163720 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.527+0000 7fa866000640 1 -- 172.21.15.106:0/3930714563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858032440 con 0x7fa868163720 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.527+0000 7fa866000640 1 -- 172.21.15.106:0/3930714563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858032810 con 0x7fa868163720 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 -- 172.21.15.106:0/3930714563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 msgr2=0x7fa86806f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 --2- 172.21.15.106:0/3930714563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa86806f390 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa858009870 tx=0x7fa85802a180 comp rx=0 tx=0).stop 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 -- 172.21.15.106:0/3930714563 shutdown_connections 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 --2- 172.21.15.106:0/3930714563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa86806e430 0x7fa8680b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 --2- 172.21.15.106:0/3930714563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa86806f390 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 -- 172.21.15.106:0/3930714563 >> 172.21.15.106:0/3930714563 conn(0x7fa8680d0c80 msgr2=0x7fa8680d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 -- 172.21.15.106:0/3930714563 shutdown_connections 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.528+0000 7fa86de00640 1 -- 172.21.15.106:0/3930714563 wait complete. 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.530+0000 7fa86de00640 1 Processor -- start 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.530+0000 7fa86de00640 1 -- start start 2024-04-26T21:47:31.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86806e430 0x7fa868154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 0x7fa868166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680cdb00 con 0x7fa868163720 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680b53f0 con 0x7fa86806e430 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa86de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680cdc80 con 0x7fa8680cd6a0 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54990/0 (socket says 172.21.15.106:54990) 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 -- 172.21.15.106:0/1642954639 learned_addr learned my addr 172.21.15.106:0/1642954639 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 -- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 msgr2=0x7fa868166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa85fe00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 0x7fa868166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 0x7fa868166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa867400640 1 -- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86806e430 msgr2=0x7fa868154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.531+0000 7fa866a00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86806e430 0x7fa868154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa867400640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86806e430 0x7fa868154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa867400640 1 -- 172.21.15.106:0/1642954639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa868160560 con 0x7fa868163720 2024-04-26T21:47:31.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa85fe00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 0x7fa868166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:31.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa867400640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c780 tx=0x7fa86000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860017070 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860003da0 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600052b0 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa868160840 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.532+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa868160e00 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.533+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868069540 con 0x7fa868163720 2024-04-26T21:47:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.534+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa86001a070 con 0x7fa868163720 2024-04-26T21:47:31.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.535+0000 7fa85f400640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 0x7fa84406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.535+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa86008b260 con 0x7fa868163720 2024-04-26T21:47:31.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.536+0000 7fa866a00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 0x7fa84406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.536+0000 7fa866a00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 0x7fa84406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa85802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.540+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8600595f0 con 0x7fa868163720 2024-04-26T21:47:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:31.782+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa8680b0d90 con 0x7fa868163720 2024-04-26T21:47:32.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.577+0000 7fa85f400640 1 -- 172.21.15.106:0/1642954639 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7fa860058ef0 con 0x7fa868163720 2024-04-26T21:47:32.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 msgr2=0x7fa84406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 0x7fa84406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa85802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 msgr2=0x7fa868167230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c780 tx=0x7fa86000cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 shutdown_connections 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa868163720 0x7fa868167230 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa84406c8f0 0x7fa84406edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680cd6a0 0x7fa868166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 --2- 172.21.15.106:0/1642954639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86806e430 0x7fa868154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 >> 172.21.15.106:0/1642954639 conn(0x7fa8680d0c80 msgr2=0x7fa8680cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.581+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 shutdown_connections 2024-04-26T21:47:32.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.582+0000 7fa86de00640 1 -- 172.21.15.106:0/1642954639 wait complete. 2024-04-26T21:47:32.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T21:47:32.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.798+0000 7f6793000640 1 Processor -- start 2024-04-26T21:47:32.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.798+0000 7f6793000640 1 -- start start 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6793000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f678c0d5990 0x7f678c163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f678c0b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6793000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c0d3f50 con 0x7f678c15f1b0 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6793000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f678c0d36b0 con 0x7f678c0d5990 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f678c0d3990 con 0x7f678c0b82b0 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6791c00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f678c15f1b0 0x7f678c162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54308/0 (socket says 172.21.15.106:54308) 2024-04-26T21:47:32.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.800+0000 7f6791c00640 1 -- 172.21.15.106:0/1892951579 learned_addr learned my addr 172.21.15.106:0/1892951579 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:32.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f6791200640 1 --2- 172.21.15.106:0/1892951579 >> v2:172.21.15.204:3300/0 conn(0x7f678c0d5990 0x7f678c163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253664705 0 0) 0x7f678c0d3f50 con 0x7f678c15f1b0 2024-04-26T21:47:32.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f6792600640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f678c0b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6774003520 con 0x7f678c15f1b0 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634546378 0 0) 0x7f6774003520 con 0x7f678c15f1b0 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 msgr2=0x7f678c0b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f678c0b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 >> v2:172.21.15.204:3300/0 conn(0x7f678c0d5990 msgr2=0x7f678c163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 --2- 172.21.15.106:0/1892951579 >> v2:172.21.15.204:3300/0 conn(0x7f678c0d5990 0x7f678c163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c0d3c70 con 0x7f678c15f1b0 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f6780001670 con 0x7f678c15f1b0 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f6791200640 1 --2- 172.21.15.106:0/1892951579 >> v2:172.21.15.204:3300/0 conn(0x7f678c0d5990 0x7f678c163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 >> v1:172.21.15.106:6789/0 conn(0x7f678c15f1b0 legacy=0x7f678c162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.801+0000 7f678be00640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f6774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f678be00640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6774004d40 0x7f67740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f678c0d36b0 con 0x7f6774004d40 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f678c0d3990 con 0x7f67740039b0 2024-04-26T21:47:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f677400b9f0 con 0x7f678c0b82b0 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6774004d40 0x7f67740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6791c00640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6791200640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f6774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 -- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 msgr2=0x7f6774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f6774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 -- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67740039b0 msgr2=0x7f6774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6792600640 1 -- 172.21.15.106:0/1892951579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6774003520 con 0x7f6774004d40 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.802+0000 7f6791200640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0b82b0 0x7f6774004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.803+0000 7f6792600640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6774004d40 0x7f67740051c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ac0 tx=0x7f6774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc825a4671e5ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784036070 con 0x7f6774004d40 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784031400 con 0x7f6774004d40 2024-04-26T21:47:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f678be00640 1 -- 172.21.15.106:0/1892951579 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784009690 con 0x7f6774004d40 2024-04-26T21:47:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 -- 172.21.15.106:0/1892951579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6774004d40 msgr2=0x7f67740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 --2- 172.21.15.106:0/1892951579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6774004d40 0x7f67740051c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ac0 tx=0x7f6774005830 comp rx=0 tx=0).stop 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6792600640 1 -- 172.21.15.106:0/1892951579 reap_dead start 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 -- 172.21.15.106:0/1892951579 shutdown_connections 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 -- 172.21.15.106:0/1892951579 >> 172.21.15.106:0/1892951579 conn(0x7f678c09e230 msgr2=0x7f678c0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 -- 172.21.15.106:0/1892951579 shutdown_connections 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.804+0000 7f6793000640 1 -- 172.21.15.106:0/1892951579 wait complete. 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.806+0000 7f6793000640 1 Processor -- start 2024-04-26T21:47:32.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.806+0000 7f6793000640 1 -- start start 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774004d40 0x7f678c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15bf90 0x7f678c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f678c163750 con 0x7f678c0ce2a0 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f678c0d33a0 con 0x7f6774004d40 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.807+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f678c1638d0 con 0x7f678c15bf90 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55014/0 (socket says 172.21.15.106:55014) 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 -- 172.21.15.106:0/1037397660 learned_addr learned my addr 172.21.15.106:0/1037397660 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6792600640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15bf90 0x7f678c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 -- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15bf90 msgr2=0x7f678c15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15bf90 0x7f678c15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791c00640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774004d40 0x7f678c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 -- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774004d40 msgr2=0x7f678c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774004d40 0x7f678c0cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 -- 172.21.15.106:0/1037397660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c15c950 con 0x7f678c0ce2a0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.808+0000 7f6791200640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e470 tx=0x7f677c00e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.809+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0137d0 con 0x7f678c0ce2a0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.809+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c01a070 con 0x7f678c0ce2a0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.809+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c013bb0 con 0x7f678c0ce2a0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.809+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c0ce6e0 con 0x7f678c0ce2a0 2024-04-26T21:47:32.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.809+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c0cc8a0 con 0x7f678c0ce2a0 2024-04-26T21:47:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.810+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c0693c0 con 0x7f678c0ce2a0 2024-04-26T21:47:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.811+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f677c012030 con 0x7f678c0ce2a0 2024-04-26T21:47:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.812+0000 7f6789600640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 0x7f677006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.813+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f677c08c920 con 0x7f678c0ce2a0 2024-04-26T21:47:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.813+0000 7f6791c00640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 0x7f677006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.813+0000 7f6791c00640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 0x7f677006ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6780001050 tx=0x7f6780005c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:32.817+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f677c010040 con 0x7f678c0ce2a0 2024-04-26T21:47:33.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.043+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f678c08d090 con 0x7f677006c920 2024-04-26T21:47:33.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.046+0000 7f6789600640 1 -- 172.21.15.106:0/1037397660 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f678c08d090 con 0x7f677006c920 2024-04-26T21:47:33.046 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-26T21:47:33.046 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:33.046 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2024-04-26T21:47:33.047 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:33.047 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 msgr2=0x7f677006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 0x7f677006ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6780001050 tx=0x7f6780005c80 comp rx=0 tx=0).stop 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 msgr2=0x7f678c06d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e470 tx=0x7f677c00e930 comp rx=0 tx=0).stop 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 shutdown_connections 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f677006c920 0x7f677006ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15bf90 0x7f678c15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678c0ce2a0 0x7f678c06d0b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 --2- 172.21.15.106:0/1037397660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774004d40 0x7f678c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 >> 172.21.15.106:0/1037397660 conn(0x7f678c09e230 msgr2=0x7f678c0b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 shutdown_connections 2024-04-26T21:47:33.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.049+0000 7f6793000640 1 -- 172.21.15.106:0/1037397660 wait complete. 2024-04-26T21:47:33.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:47:33.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.272+0000 7f7e1be00640 1 Processor -- start 2024-04-26T21:47:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.273+0000 7f7e1be00640 1 -- start start 2024-04-26T21:47:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.274+0000 7f7e1be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e1c15ee60 0x7f7e1c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.274+0000 7f7e1be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7e1c15cf40 0x7f7e1c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.274+0000 7f7e1be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e1c15d970 con 0x7f7e1c15c450 2024-04-26T21:47:33.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.274+0000 7f7e1be00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1c06e520 con 0x7f7e1c15ee60 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.274+0000 7f7e1be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1c0b4fe0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e1c15ee60 0x7f7e1c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e1c15ee60 0x7f7e1c0b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55020/0 (socket says 172.21.15.106:55020) 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 -- 172.21.15.106:0/398559794 learned_addr learned my addr 172.21.15.106:0/398559794 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e1b400640 1 --2- 172.21.15.106:0/398559794 >> v2:172.21.15.204:3300/0 conn(0x7f7e1c15cf40 0x7f7e1c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 -- 172.21.15.106:0/398559794 >> v2:172.21.15.204:3300/0 conn(0x7f7e1c15cf40 msgr2=0x7f7e1c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 --2- 172.21.15.106:0/398559794 >> v2:172.21.15.204:3300/0 conn(0x7f7e1c15cf40 0x7f7e1c06dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 -- 172.21.15.106:0/398559794 >> v1:172.21.15.204:6789/0 conn(0x7f7e1c15c450 legacy=0x7f7e1c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:33.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.275+0000 7f7e13e00640 1 -- 172.21.15.106:0/398559794 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c0b52c0 con 0x7f7e1c15ee60 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.276+0000 7f7e13e00640 1 --2- 172.21.15.106:0/398559794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15ee60 0x7f7e1c0b4920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0ccf30 tx=0x7f7e0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d173228a99e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1a000640 1 -- 172.21.15.106:0/398559794 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04037070 con 0x7f7e1c15ee60 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1a000640 1 -- 172.21.15.106:0/398559794 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e04032440 con 0x7f7e1c15ee60 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1a000640 1 -- 172.21.15.106:0/398559794 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04032810 con 0x7f7e1c15ee60 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 -- 172.21.15.106:0/398559794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15ee60 msgr2=0x7f7e1c0b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 --2- 172.21.15.106:0/398559794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15ee60 0x7f7e1c0b4920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0ccf30 tx=0x7f7e0402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 -- 172.21.15.106:0/398559794 shutdown_connections 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 --2- 172.21.15.106:0/398559794 >> v2:172.21.15.204:3300/0 conn(0x7f7e1c15cf40 0x7f7e1c06dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 --2- 172.21.15.106:0/398559794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15ee60 0x7f7e1c0b4920 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.277+0000 7f7e1be00640 1 -- 172.21.15.106:0/398559794 >> 172.21.15.106:0/398559794 conn(0x7f7e1c0b2900 msgr2=0x7f7e1c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:33.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.278+0000 7f7e1be00640 1 -- 172.21.15.106:0/398559794 shutdown_connections 2024-04-26T21:47:33.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.278+0000 7f7e1be00640 1 -- 172.21.15.106:0/398559794 wait complete. 2024-04-26T21:47:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.280+0000 7f7e1be00640 1 Processor -- start 2024-04-26T21:47:33.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.280+0000 7f7e1be00640 1 -- start start 2024-04-26T21:47:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e1c15c450 0x7f7e1c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e1c15ee60 0x7f7e1c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0b5a00 con 0x7f7e1c15cf40 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0b5880 con 0x7f7e1c15c450 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e1be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1c165a00 con 0x7f7e1c15ee60 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e13e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e13e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55024/0 (socket says 172.21.15.106:55024) 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.281+0000 7f7e13e00640 1 -- 172.21.15.106:0/3825375900 learned_addr learned my addr 172.21.15.106:0/3825375900 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 -- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e1c15ee60 msgr2=0x7f7e1c073660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e1c15ee60 0x7f7e1c073660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 -- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e1c15c450 msgr2=0x7f7e1c153a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:47:33.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e1c15c450 0x7f7e1c153a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 -- 172.21.15.106:0/3825375900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c1494e0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e13e00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7e04000c00 tx=0x7f7e040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04041020 con 0x7f7e1c15cf40 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e040306c0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c1497c0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04037040 con 0x7f7e1c15cf40 2024-04-26T21:47:33.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.282+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c149d80 con 0x7f7e1c15cf40 2024-04-26T21:47:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.284+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c0693c0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.285+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e04044020 con 0x7f7e1c15cf40 2024-04-26T21:47:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.286+0000 7f7e12a00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 0x7f7df806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.286+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7e040ac390 con 0x7f7e1c15cf40 2024-04-26T21:47:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.286+0000 7f7e1aa00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 0x7f7df806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.287+0000 7f7e1aa00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 0x7f7df806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c003c50 tx=0x7f7e0c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.291+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e0404b030 con 0x7f7e1c15cf40 2024-04-26T21:47:33.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.492+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7e1c0d0da0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.493+0000 7f7e12a00640 1 -- 172.21.15.106:0/3825375900 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f7e0407a7b0 con 0x7f7e1c15cf40 2024-04-26T21:47:33.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 msgr2=0x7f7df806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 0x7f7df806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c003c50 tx=0x7f7e0c0079b0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 msgr2=0x7f7e1c14a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7e04000c00 tx=0x7f7e040046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 shutdown_connections 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7df806c920 0x7f7df806ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e1c15ee60 0x7f7e1c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e1c15cf40 0x7f7e1c14a2a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 --2- 172.21.15.106:0/3825375900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e1c15c450 0x7f7e1c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 >> 172.21.15.106:0/3825375900 conn(0x7f7e1c0b2900 msgr2=0x7f7e1c06f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 shutdown_connections 2024-04-26T21:47:33.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:33.497+0000 7f7e1be00640 1 -- 172.21.15.106:0/3825375900 wait complete. 2024-04-26T21:47:33.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:33.522 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp28350/subvol14680) 2024-04-26T21:47:33.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:33.626 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-26T21:47:33.679 INFO:teuthology.orchestra.run.smithi106.stdout:cephfs_data 2024-04-26T21:47:33.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:33.680 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp28350/subvol14680) 2024-04-26T21:47:33.766 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:33.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp28350/subvol14680/.meta 2024-04-26T21:47:33.862 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol14680 subvol_grp28350 2024-04-26T21:47:34.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.061+0000 7fae86200640 1 Processor -- start 2024-04-26T21:47:34.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.061+0000 7fae86200640 1 -- start start 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae86200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae80072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae86200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae80159af0 con 0x7fae80073180 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae86200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae80072180 con 0x7fae800b3c30 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae86200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae80160210 con 0x7fae80159710 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae76a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fae80073180 0x7fae8015f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54314/0 (socket says 172.21.15.106:54314) 2024-04-26T21:47:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.063+0000 7fae76a00640 1 -- 172.21.15.106:0/445106700 learned_addr learned my addr 172.21.15.106:0/445106700 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7f400640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae80072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046796305 0 0) 0x7fae80159af0 con 0x7fae80073180 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae80073180 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994734934 0 0) 0x7fae64003520 con 0x7fae80073180 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 msgr2=0x7fae80072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae80072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 >> v1:172.21.15.204:6789/0 conn(0x7fae800b3c30 legacy=0x7fae800738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae801604f0 con 0x7fae80073180 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fae68001670 con 0x7fae80073180 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 >> v1:172.21.15.106:6789/0 conn(0x7fae80073180 legacy=0x7fae8015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.064+0000 7fae7ea00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae640039e0 0x7fae64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7ea00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7ea00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae64004d40 0x7fae640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fae80160210 con 0x7fae64004d40 2024-04-26T21:47:34.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae6400ba70 con 0x7fae640039e0 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400bc70 con 0x7fae80159710 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae64004d40 0x7fae640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae76a00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7f400640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae640039e0 0x7fae64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 -- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 msgr2=0x7fae64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae80159710 0x7fae64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 -- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae640039e0 msgr2=0x7fae64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae640039e0 0x7fae64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.065+0000 7fae7fe00640 1 -- 172.21.15.106:0/445106700 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003520 con 0x7fae64004d40 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.066+0000 7fae7fe00640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae64004d40 0x7fae640051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae78001e20 tx=0x7fae64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134247fd7bffdd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78001a70 con 0x7fae64004d40 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7802ae40 con 0x7fae64004d40 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae7ea00640 1 -- 172.21.15.106:0/445106700 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78003bb0 con 0x7fae64004d40 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 -- 172.21.15.106:0/445106700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae64004d40 msgr2=0x7fae640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 --2- 172.21.15.106:0/445106700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae64004d40 0x7fae640051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae78001e20 tx=0x7fae64005830 comp rx=0 tx=0).stop 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae7fe00640 1 -- 172.21.15.106:0/445106700 reap_dead start 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 -- 172.21.15.106:0/445106700 shutdown_connections 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 -- 172.21.15.106:0/445106700 >> 172.21.15.106:0/445106700 conn(0x7fae800d0c40 msgr2=0x7fae800d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 -- 172.21.15.106:0/445106700 shutdown_connections 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.067+0000 7fae86200640 1 -- 172.21.15.106:0/445106700 wait complete. 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.069+0000 7fae86200640 1 Processor -- start 2024-04-26T21:47:34.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.069+0000 7fae86200640 1 -- start start 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae64004d40 0x7fae80151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae8014eb50 0x7fae800b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fae80070130 con 0x7fae801516d0 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae80072720 con 0x7fae8014eb50 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.070+0000 7fae86200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae8015fdf0 con 0x7fae64004d40 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55042/0 (socket says 172.21.15.106:55042) 2024-04-26T21:47:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 -- 172.21.15.106:0/3122239062 learned_addr learned my addr 172.21.15.106:0/3122239062 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7f400640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae64004d40 0x7fae80151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae76a00640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae8014eb50 0x7fae800b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 -- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae64004d40 msgr2=0x7fae80151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae64004d40 0x7fae80151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 -- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae8014eb50 msgr2=0x7fae800b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae8014eb50 0x7fae800b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 -- 172.21.15.106:0/3122239062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8009c660 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.071+0000 7fae7fe00640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae78001f40 tx=0x7fae780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.072+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78042070 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.072+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae78006b50 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.072+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8009c940 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.072+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78037040 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.072+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae800b18c0 con 0x7fae801516d0 2024-04-26T21:47:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.073+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae800b5ee0 con 0x7fae801516d0 2024-04-26T21:47:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.074+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae78045070 con 0x7fae801516d0 2024-04-26T21:47:34.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.075+0000 7fae77e00640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 0x7fae5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.075+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fae780ab2b0 con 0x7fae801516d0 2024-04-26T21:47:34.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.075+0000 7fae7f400640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 0x7fae5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.076+0000 7fae7f400640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 0x7fae5c06ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae70009a10 tx=0x7fae70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.079+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae780795b0 con 0x7fae801516d0 2024-04-26T21:47:34.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.307+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol14680", "group_name": "subvol_grp28350", "target": ["mon-mgr", ""]}) -- 0x7fae80069650 con 0x7fae5c06c920 2024-04-26T21:47:34.315 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:34.315+0000 7fe623000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:34.315 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:34.315+0000 7fe623000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:34.315 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:34.315+0000 7fe623000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:34.315 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:34.315+0000 7fe623000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:34.315 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:34.315+0000 7fe623000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:34.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.388+0000 7fae77e00640 1 -- 172.21.15.106:0/3122239062 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae80069650 con 0x7fae5c06c920 2024-04-26T21:47:34.388 INFO:teuthology.orchestra.run.smithi106.stdout:/volumes/subvol_grp28350/subvol14680 2024-04-26T21:47:34.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 msgr2=0x7fae5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 0x7fae5c06ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae70009a10 tx=0x7fae70008040 comp rx=0 tx=0).stop 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 msgr2=0x7fae800b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae78001f40 tx=0x7fae780014a0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 shutdown_connections 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae801516d0 0x7fae800b2320 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae8014eb50 0x7fae800b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fae5c06c920 0x7fae5c06ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 --2- 172.21.15.106:0/3122239062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae64004d40 0x7fae80151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 >> 172.21.15.106:0/3122239062 conn(0x7fae800d0c40 msgr2=0x7fae80150920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 shutdown_connections 2024-04-26T21:47:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.391+0000 7fae86200640 1 -- 172.21.15.106:0/3122239062 wait complete. 2024-04-26T21:47:34.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol14680 subvol_grp28350 2024-04-26T21:47:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.610+0000 7f1dac800640 1 Processor -- start 2024-04-26T21:47:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.610+0000 7f1dac800640 1 -- start start 2024-04-26T21:47:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1dac800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1da41576b0 0x7f1da4073da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1dac800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1da40742e0 0x7f1da406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1dac800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da4157a90 con 0x7f1da406c890 2024-04-26T21:47:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1dac800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1da40af230 con 0x7f1da41576b0 2024-04-26T21:47:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1dac800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1da415e2e0 con 0x7f1da40742e0 2024-04-26T21:47:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1da9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1da41576b0 0x7f1da4073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1da9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1da41576b0 0x7f1da4073da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55074/0 (socket says 172.21.15.106:55074) 2024-04-26T21:47:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.611+0000 7f1da9e00640 1 -- 172.21.15.106:0/3684380267 learned_addr learned my addr 172.21.15.106:0/3684380267 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.612+0000 7f1da9e00640 1 -- 172.21.15.106:0/3684380267 >> v2:172.21.15.204:3300/0 conn(0x7f1da40742e0 msgr2=0x7f1da406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.612+0000 7f1da9e00640 1 --2- 172.21.15.106:0/3684380267 >> v2:172.21.15.204:3300/0 conn(0x7f1da40742e0 0x7f1da406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.612+0000 7f1da9e00640 1 -- 172.21.15.106:0/3684380267 >> v1:172.21.15.204:6789/0 conn(0x7f1da406c890 legacy=0x7f1da415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.612+0000 7f1da9e00640 1 -- 172.21.15.106:0/3684380267 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da40732c0 con 0x7f1da41576b0 2024-04-26T21:47:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.613+0000 7f1da9e00640 1 --2- 172.21.15.106:0/3684380267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da41576b0 0x7f1da4073da0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1da40d13a0 tx=0x7f1d9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccd160e2593e604 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.613+0000 7f1da8a00640 1 -- 172.21.15.106:0/3684380267 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98037070 con 0x7f1da41576b0 2024-04-26T21:47:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.613+0000 7f1da8a00640 1 -- 172.21.15.106:0/3684380267 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d98032440 con 0x7f1da41576b0 2024-04-26T21:47:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.613+0000 7f1da8a00640 1 -- 172.21.15.106:0/3684380267 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98032810 con 0x7f1da41576b0 2024-04-26T21:47:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 -- 172.21.15.106:0/3684380267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da41576b0 msgr2=0x7f1da4073da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 --2- 172.21.15.106:0/3684380267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da41576b0 0x7f1da4073da0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1da40d13a0 tx=0x7f1d9802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 -- 172.21.15.106:0/3684380267 shutdown_connections 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 --2- 172.21.15.106:0/3684380267 >> v2:172.21.15.204:3300/0 conn(0x7f1da40742e0 0x7f1da406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 --2- 172.21.15.106:0/3684380267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da41576b0 0x7f1da4073da0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 -- 172.21.15.106:0/3684380267 >> 172.21.15.106:0/3684380267 conn(0x7f1da40b3c00 msgr2=0x7f1da40b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 -- 172.21.15.106:0/3684380267 shutdown_connections 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.614+0000 7f1dac800640 1 -- 172.21.15.106:0/3684380267 wait complete. 2024-04-26T21:47:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.616+0000 7f1dac800640 1 Processor -- start 2024-04-26T21:47:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.616+0000 7f1dac800640 1 -- start start 2024-04-26T21:47:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da40742e0 0x7f1da415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1da41576b0 0x7f1da4157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1da415dd70 con 0x7f1da406c890 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4073880 con 0x7f1da40742e0 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1dac800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1da415def0 con 0x7f1da41576b0 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55076/0 (socket says 172.21.15.106:55076) 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 -- 172.21.15.106:0/1435149767 learned_addr learned my addr 172.21.15.106:0/1435149767 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 -- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1da41576b0 msgr2=0x7f1da4157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1da41576b0 0x7f1da4157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 -- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da40742e0 msgr2=0x7f1da415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da40742e0 0x7f1da415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.617+0000 7f1da9e00640 1 -- 172.21.15.106:0/1435149767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da415e520 con 0x7f1da406c890 2024-04-26T21:47:34.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1da9e00640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d98000c00 tx=0x7f1d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98041070 con 0x7f1da406c890 2024-04-26T21:47:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9803c870 con 0x7f1da406c890 2024-04-26T21:47:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98037040 con 0x7f1da406c890 2024-04-26T21:47:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da415e800 con 0x7f1da406c890 2024-04-26T21:47:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.618+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da415edc0 con 0x7f1da406c890 2024-04-26T21:47:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.619+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da40ce410 con 0x7f1da406c890 2024-04-26T21:47:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.620+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d98044070 con 0x7f1da406c890 2024-04-26T21:47:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.621+0000 7f1d9e000640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 0x7f1d8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.621+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1d980ac6d0 con 0x7f1da406c890 2024-04-26T21:47:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.621+0000 7f1da9400640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 0x7f1d8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.622+0000 7f1da9400640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 0x7f1d8806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1da40af1b0 tx=0x7f1d94006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:34.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.625+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d9807aaf0 con 0x7f1da406c890 2024-04-26T21:47:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.860+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol14680", "group_name": "subvol_grp28350", "target": ["mon-mgr", ""]}) -- 0x7f1da4069650 con 0x7f1d8806c920 2024-04-26T21:47:34.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.870+0000 7f1d9e000640 1 -- 172.21.15.106:0/1435149767 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1da4069650 con 0x7f1d8806c920 2024-04-26T21:47:34.870 INFO:teuthology.orchestra.run.smithi106.stdout:/volumes/subvol_grp28350/subvol14680 2024-04-26T21:47:34.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 msgr2=0x7f1d8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 0x7f1d8806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1da40af1b0 tx=0x7f1d94006c90 comp rx=0 tx=0).stop 2024-04-26T21:47:34.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 msgr2=0x7f1da4147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:34.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d98000c00 tx=0x7f1d980046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 shutdown_connections 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1d8806c920 0x7f1d8806ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1da41576b0 0x7f1da4157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da40742e0 0x7f1da415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 --2- 172.21.15.106:0/1435149767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1da406c890 0x7f1da4147850 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 >> 172.21.15.106:0/1435149767 conn(0x7f1da40b3c00 msgr2=0x7f1da40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 shutdown_connections 2024-04-26T21:47:34.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:34.873+0000 7f1dac800640 1 -- 172.21.15.106:0/1435149767 wait complete. 2024-04-26T21:47:34.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:34.893 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/3caf5d2116af1d5f80dd323b0871adde.meta) 2024-04-26T21:47:36.659 INFO:teuthology.orchestra.run.smithi106.stdout:[GLOBAL] 2024-04-26T21:47:36.660 INFO:teuthology.orchestra.run.smithi106.stdout:version = 1 2024-04-26T21:47:36.660 INFO:teuthology.orchestra.run.smithi106.stdout:type = subvolume 2024-04-26T21:47:36.660 INFO:teuthology.orchestra.run.smithi106.stdout:path = /volumes/subvol_grp28350/subvol14680 2024-04-26T21:47:36.660 INFO:teuthology.orchestra.run.smithi106.stdout:state = complete 2024-04-26T21:47:36.660 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:36.666 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol14680 subvol_grp28350 2024-04-26T21:47:36.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.876+0000 7fc1e4c00640 1 Processor -- start 2024-04-26T21:47:36.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.877+0000 7fc1e4c00640 1 -- start start 2024-04-26T21:47:36.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.878+0000 7fc1e4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc1e00cffe0 0x7fc1e00af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.878+0000 7fc1e4c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e00ae160 con 0x7fc1e00afd40 2024-04-26T21:47:36.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.878+0000 7fc1e4c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e006ff80 con 0x7fc1e015d170 2024-04-26T21:47:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.878+0000 7fc1e4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e015d550 con 0x7fc1e00cffe0 2024-04-26T21:47:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.879+0000 7fc1de800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc1e00afd40 0x7fc1e0072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54330/0 (socket says 172.21.15.106:54330) 2024-04-26T21:47:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.879+0000 7fc1de800640 1 -- 172.21.15.106:0/3233618342 learned_addr learned my addr 172.21.15.106:0/3233618342 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.879+0000 7fc1d5400640 1 --2- 172.21.15.106:0/3233618342 >> v2:172.21.15.204:3300/0 conn(0x7fc1e00cffe0 0x7fc1e00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.879+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585850438 0 0) 0x7fc1e00ae160 con 0x7fc1e00afd40 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.879+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c4003520 con 0x7fc1e00afd40 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3262255817 0 0) 0x7fc1c4003520 con 0x7fc1e00afd40 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 >> v2:172.21.15.204:3300/0 conn(0x7fc1e00cffe0 msgr2=0x7fc1e00af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 --2- 172.21.15.106:0/3233618342 >> v2:172.21.15.204:3300/0 conn(0x7fc1e00cffe0 0x7fc1e00af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 >> v1:172.21.15.204:6789/0 conn(0x7fc1e015d170 legacy=0x7fc1e006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e0070260 con 0x7fc1e00afd40 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fc1d80019f0 con 0x7fc1e00afd40 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 >> v1:172.21.15.106:6789/0 conn(0x7fc1e00afd40 legacy=0x7fc1e0072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c4004800 0x7fc1c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c40051c0 0x7fc1c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e015d550 con 0x7fc1c40039b0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c400c060 con 0x7fc1c40051c0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c400c260 con 0x7fc1c4004800 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.880+0000 7fc1dde00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1de800640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c40051c0 0x7fc1c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1d5400640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c4004800 0x7fc1c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1dde00640 1 -- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c4004800 msgr2=0x7fc1c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1dde00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c4004800 0x7fc1c4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1dde00640 1 -- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c40051c0 msgr2=0x7fc1c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1dde00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c40051c0 0x7fc1c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1dde00640 1 -- 172.21.15.106:0/3233618342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4003520 con 0x7fc1c40039b0 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1de800640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c40051c0 0x7fc1c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:36.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.881+0000 7fc1de800640 1 -- 172.21.15.106:0/3233618342 reap_dead start 2024-04-26T21:47:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.882+0000 7fc1dde00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1c4003dd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0001750 tx=0x7fc1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7d8b67d8a0e604 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.882+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0001370 con 0x7fc1c40039b0 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.882+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d00052d0 con 0x7fc1c40039b0 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.882+0000 7fc1dd400640 1 -- 172.21.15.106:0/3233618342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d002abc0 con 0x7fc1c40039b0 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.882+0000 7fc1e4c00640 1 -- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 msgr2=0x7fc1c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1c4003dd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0001750 tx=0x7fc1c4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 -- 172.21.15.106:0/3233618342 shutdown_connections 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/3233618342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1c4003dd0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 -- 172.21.15.106:0/3233618342 >> 172.21.15.106:0/3233618342 conn(0x7fc1e00d0c40 msgr2=0x7fc1e00d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 -- 172.21.15.106:0/3233618342 shutdown_connections 2024-04-26T21:47:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.883+0000 7fc1e4c00640 1 -- 172.21.15.106:0/3233618342 wait complete. 2024-04-26T21:47:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.884+0000 7fc1e4c00640 1 Processor -- start 2024-04-26T21:47:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.885+0000 7fc1e4c00640 1 -- start start 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.885+0000 7fc1e4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.885+0000 7fc1e4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 0x7fc1e009c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1e4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1e00ce2b0 0x7fc1e009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1e4c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e00d0440 con 0x7fc1c40039b0 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1e4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e006fd30 con 0x7fc1e00cde70 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1e4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00b0170 con 0x7fc1e00ce2b0 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55096/0 (socket says 172.21.15.106:55096) 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 -- 172.21.15.106:0/1978155518 learned_addr learned my addr 172.21.15.106:0/1978155518 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1de800640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1e00ce2b0 0x7fc1e009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1d5400640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 0x7fc1e009c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 -- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1e00ce2b0 msgr2=0x7fc1e009cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1e00ce2b0 0x7fc1e009cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 -- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 msgr2=0x7fc1e009c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 0x7fc1e009c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1dde00640 1 -- 172.21.15.106:0/1978155518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e0156440 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.886+0000 7fc1d5400640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 0x7fc1e009c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1dde00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005040 tx=0x7fc1d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0037040 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d0040070 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0007a30 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e0156720 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.887+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e0156ce0 con 0x7fc1c40039b0 2024-04-26T21:47:36.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.888+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0069650 con 0x7fc1c40039b0 2024-04-26T21:47:36.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.889+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc1d00052d0 con 0x7fc1c40039b0 2024-04-26T21:47:36.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.890+0000 7fc1d6a00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 0x7fc1bc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:36.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.890+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc1d00ab8e0 con 0x7fc1c40039b0 2024-04-26T21:47:36.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.890+0000 7fc1d5400640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 0x7fc1bc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:36.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.891+0000 7fc1d5400640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 0x7fc1bc06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1e01579b0 tx=0x7fc1c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:36.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:36.895+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1d0079c90 con 0x7fc1c40039b0 2024-04-26T21:47:37.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.117+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol14680", "group_name": "subvol_grp28350", "target": ["mon-mgr", ""]}) -- 0x7fc1e00b0550 con 0x7fc1bc06c980 2024-04-26T21:47:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.163+0000 7fc1d6a00640 1 -- 172.21.15.106:0/1978155518 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1e00b0550 con 0x7fc1bc06c980 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.165+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 msgr2=0x7fc1bc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.165+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 0x7fc1bc06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1e01579b0 tx=0x7fc1c8008040 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.165+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 msgr2=0x7fc1e01518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.165+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005040 tx=0x7fc1d0001120 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 shutdown_connections 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc1bc06c980 0x7fc1bc06ee40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1e00ce2b0 0x7fc1e009cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e00cde70 0x7fc1e009c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 --2- 172.21.15.106:0/1978155518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1c40039b0 0x7fc1e01518f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 >> 172.21.15.106:0/1978155518 conn(0x7fc1e00d0c40 msgr2=0x7fc1e00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 shutdown_connections 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.166+0000 7fc1e4c00640 1 -- 172.21.15.106:0/1978155518 wait complete. 2024-04-26T21:47:37.170 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.169+0000 7fe625800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.170 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.169+0000 7fe625800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.170 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.169+0000 7fe625800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.170 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.169+0000 7fe625800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.170 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.169+0000 7fe625800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.186 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-26T21:47:37.196 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.195+0000 7fe626200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.196 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.195+0000 7fe626200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.196 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.195+0000 7fe626200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.196 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.195+0000 7fe626200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.196 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-26T21:47:37.195+0000 7fe626200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:47:37.281 INFO:teuthology.orchestra.run.smithi106.stdout:2 2024-04-26T21:47:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:37.290 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp28350 2024-04-26T21:47:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.531+0000 7ffa20c00640 1 Processor -- start 2024-04-26T21:47:37.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.531+0000 7ffa20c00640 1 -- start start 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa20c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 0x7ffa1c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa20c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0af800 con 0x7ffa1c15d100 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa20c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0ae340 con 0x7ffa1c0b5ca0 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa20c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0afaa0 con 0x7ffa1c06ce80 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa19e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffa1c15d100 0x7ffa1c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54340/0 (socket says 172.21.15.106:54340) 2024-04-26T21:47:37.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa19e00640 1 -- 172.21.15.106:0/599669617 learned_addr learned my addr 172.21.15.106:0/599669617 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 895565107 0 0) 0x7ffa1c0af800 con 0x7ffa1c15d100 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.533+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa00003520 con 0x7ffa1c15d100 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa19400640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 0x7ffa1c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2941684176 0 0) 0x7ffa00003520 con 0x7ffa1c15d100 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 msgr2=0x7ffa1c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 0x7ffa1c0b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 >> v1:172.21.15.204:6789/0 conn(0x7ffa1c0b5ca0 legacy=0x7ffa1c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0afd80 con 0x7ffa1c15d100 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7ffa0c001670 con 0x7ffa1c15d100 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 >> v1:172.21.15.106:6789/0 conn(0x7ffa1c15d100 legacy=0x7ffa1c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa000039e0 0x7ffa00003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 0x7ffa00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa00004d40 0x7ffa000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0afaa0 con 0x7ffa00004d40 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0000bba0 con 0x7ffa000039e0 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0000bda0 con 0x7ffa1c06ce80 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.534+0000 7ffa1a800640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa00004d40 0x7ffa000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.535+0000 7ffa1a800640 1 -- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 msgr2=0x7ffa00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.535+0000 7ffa1a800640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06ce80 0x7ffa00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.535+0000 7ffa1a800640 1 -- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa000039e0 msgr2=0x7ffa00003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.535+0000 7ffa1a800640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa000039e0 0x7ffa00003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.535+0000 7ffa1a800640 1 -- 172.21.15.106:0/599669617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa00003520 con 0x7ffa00004d40 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa1a800640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa00004d40 0x7ffa000051c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa14001e50 tx=0x7ffa00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c161b0dae3dc393 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14001a70 con 0x7ffa00004d40 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14003970 con 0x7ffa00004d40 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa18a00640 1 -- 172.21.15.106:0/599669617 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1402ae60 con 0x7ffa00004d40 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa20c00640 1 -- 172.21.15.106:0/599669617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa00004d40 msgr2=0x7ffa000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa20c00640 1 --2- 172.21.15.106:0/599669617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa00004d40 0x7ffa000051c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa14001e50 tx=0x7ffa00005830 comp rx=0 tx=0).stop 2024-04-26T21:47:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa1a800640 1 -- 172.21.15.106:0/599669617 reap_dead start 2024-04-26T21:47:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa20c00640 1 -- 172.21.15.106:0/599669617 shutdown_connections 2024-04-26T21:47:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.536+0000 7ffa20c00640 1 -- 172.21.15.106:0/599669617 >> 172.21.15.106:0/599669617 conn(0x7ffa1c0b2d50 msgr2=0x7ffa1c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.537+0000 7ffa20c00640 1 -- 172.21.15.106:0/599669617 shutdown_connections 2024-04-26T21:47:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.537+0000 7ffa20c00640 1 -- 172.21.15.106:0/599669617 wait complete. 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.538+0000 7ffa20c00640 1 Processor -- start 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.538+0000 7ffa20c00640 1 -- start start 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 0x7ffa1c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06f840 0x7ffa1c1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c09ee50 con 0x7ffa1c15f940 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0b0340 con 0x7ffa00004d40 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa20c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c160b70 con 0x7ffa1c06f840 2024-04-26T21:47:37.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa19400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.539+0000 7ffa19400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55150/0 (socket says 172.21.15.106:55150) 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 -- 172.21.15.106:0/3404583360 learned_addr learned my addr 172.21.15.106:0/3404583360 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 -- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06f840 msgr2=0x7ffa1c1f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06f840 0x7ffa1c1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 -- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 msgr2=0x7ffa1c06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19e00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 0x7ffa1c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 0x7ffa1c06e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 -- 172.21.15.106:0/3404583360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c06e7c0 con 0x7ffa1c15f940 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19e00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 0x7ffa1c06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa19400640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d800 tx=0x7ffa0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.540+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08018020 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.541+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0800b6c0 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.541+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800ba90 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.541+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c06eaa0 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.541+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c06f060 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.542+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c069650 con 0x7ffa1c15f940 2024-04-26T21:47:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.543+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ffa0801b020 con 0x7ffa1c15f940 2024-04-26T21:47:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.544+0000 7ffa12000640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 0x7ff9fc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.544+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ffa0808bcc0 con 0x7ffa1c15f940 2024-04-26T21:47:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.544+0000 7ffa19e00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 0x7ff9fc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.544+0000 7ffa19e00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 0x7ff9fc06ee70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000ff0 tx=0x7ffa0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.548+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa08092080 con 0x7ffa1c15f940 2024-04-26T21:47:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.774+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp28350", "target": ["mon-mgr", ""]}) -- 0x7ffa1c15c7f0 con 0x7ff9fc06c9b0 2024-04-26T21:47:37.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.784+0000 7ffa12000640 1 -- 172.21.15.106:0/3404583360 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa1c15c7f0 con 0x7ff9fc06c9b0 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 msgr2=0x7ff9fc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 0x7ff9fc06ee70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000ff0 tx=0x7ffa0c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 msgr2=0x7ffa1c1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d800 tx=0x7ffa0800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 shutdown_connections 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff9fc06c9b0 0x7ff9fc06ee70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa1c06f840 0x7ffa1c1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa1c15f940 0x7ffa1c1598a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 --2- 172.21.15.106:0/3404583360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa00004d40 0x7ffa1c06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 >> 172.21.15.106:0/3404583360 conn(0x7ffa1c0b2d50 msgr2=0x7ffa1c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.786+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 shutdown_connections 2024-04-26T21:47:37.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:37.787+0000 7ffa20c00640 1 -- 172.21.15.106:0/3404583360 wait complete. 2024-04-26T21:47:37.805 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:47:37.805 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:37.906 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:38.019 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:47:38.019 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:38.021 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:38.021 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:38.076 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:38.077 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:38.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.323+0000 7fef2b200640 1 Processor -- start 2024-04-26T21:47:38.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.323+0000 7fef2b200640 1 -- start start 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef2b200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef240d0e70 0x7fef240d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef2b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fef240b5d40 0x7fef240aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef2b200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef240d1790 con 0x7fef240b5660 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef2b200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fef240727f0 con 0x7fef240d0e70 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef2b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fef240b4a10 con 0x7fef240b5d40 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef29e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef240d0e70 0x7fef240d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef29e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef240d0e70 0x7fef240d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54980/0 (socket says 172.21.15.106:54980) 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef29e00640 1 -- 172.21.15.106:0/2457913635 learned_addr learned my addr 172.21.15.106:0/2457913635 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.325+0000 7fef29400640 1 --1- 172.21.15.106:0/2457913635 >> v1:172.21.15.106:6789/0 conn(0x7fef240b5660 0x7fef240ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35602/0 (socket says 172.21.15.106:35602) 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.326+0000 7fef29e00640 1 -- 172.21.15.106:0/2457913635 >> v2:172.21.15.204:3300/0 conn(0x7fef240b5d40 msgr2=0x7fef240aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.326+0000 7fef29e00640 1 --2- 172.21.15.106:0/2457913635 >> v2:172.21.15.204:3300/0 conn(0x7fef240b5d40 0x7fef240aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.326+0000 7fef29e00640 1 -- 172.21.15.106:0/2457913635 >> v1:172.21.15.106:6789/0 conn(0x7fef240b5660 legacy=0x7fef240ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.326+0000 7fef29e00640 1 -- 172.21.15.106:0/2457913635 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef240b4cf0 con 0x7fef240d0e70 2024-04-26T21:47:38.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.327+0000 7fef29e00640 1 --2- 172.21.15.106:0/2457913635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240d0e70 0x7fef240d1250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef240720e0 tx=0x7fef1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba0c83502eec560 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.327+0000 7fef28a00640 1 -- 172.21.15.106:0/2457913635 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10037070 con 0x7fef240d0e70 2024-04-26T21:47:38.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.327+0000 7fef28a00640 1 -- 172.21.15.106:0/2457913635 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef10032440 con 0x7fef240d0e70 2024-04-26T21:47:38.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.327+0000 7fef28a00640 1 -- 172.21.15.106:0/2457913635 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10032810 con 0x7fef240d0e70 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 -- 172.21.15.106:0/2457913635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240d0e70 msgr2=0x7fef240d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 --2- 172.21.15.106:0/2457913635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240d0e70 0x7fef240d1250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef240720e0 tx=0x7fef1002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 -- 172.21.15.106:0/2457913635 shutdown_connections 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 --2- 172.21.15.106:0/2457913635 >> v2:172.21.15.204:3300/0 conn(0x7fef240b5d40 0x7fef240aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 --2- 172.21.15.106:0/2457913635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240d0e70 0x7fef240d1250 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 -- 172.21.15.106:0/2457913635 >> 172.21.15.106:0/2457913635 conn(0x7fef2406c740 msgr2=0x7fef2406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 -- 172.21.15.106:0/2457913635 shutdown_connections 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.328+0000 7fef2b200640 1 -- 172.21.15.106:0/2457913635 wait complete. 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.330+0000 7fef2b200640 1 Processor -- start 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.330+0000 7fef2b200640 1 -- start start 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b5d40 0x7fef24156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240d0e70 0x7fef241e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef240b05b0 con 0x7fef240b5660 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef240b0430 con 0x7fef240d0e70 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef2b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fef2415f1a0 con 0x7fef240b5d40 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54986/0 (socket says 172.21.15.106:54986) 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 -- 172.21.15.106:0/2269670582 learned_addr learned my addr 172.21.15.106:0/2269670582 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 -- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b5d40 msgr2=0x7fef24156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b5d40 0x7fef24156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 -- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240d0e70 msgr2=0x7fef241e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240d0e70 0x7fef241e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.331+0000 7fef29e00640 1 -- 172.21.15.106:0/2269670582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241579f0 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef29e00640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fef10000c00 tx=0x7fef100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10041020 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef10030970 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10037040 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef24157cd0 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.332+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef24158290 con 0x7fef240b5660 2024-04-26T21:47:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.333+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef240693c0 con 0x7fef240b5660 2024-04-26T21:47:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.334+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fef10044020 con 0x7fef240b5660 2024-04-26T21:47:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.336+0000 7fef1e000640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 0x7fef0406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.336+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fef100ac6c0 con 0x7fef240b5660 2024-04-26T21:47:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.336+0000 7fef29400640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 0x7fef0406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.337+0000 7fef29400640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 0x7fef0406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fef18000f60 tx=0x7fef18007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.340+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef1004b030 con 0x7fef240b5660 2024-04-26T21:47:38.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.543+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef240afd40 con 0x7fef240b5660 2024-04-26T21:47:38.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.544+0000 7fef1e000640 1 -- 172.21.15.106:0/2269670582 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7fef1007aa00 con 0x7fef240b5660 2024-04-26T21:47:38.546 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:38.546 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 msgr2=0x7fef0406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 0x7fef0406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fef18000f60 tx=0x7fef18007a00 comp rx=0 tx=0).stop 2024-04-26T21:47:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 msgr2=0x7fef24142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fef10000c00 tx=0x7fef100046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 shutdown_connections 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240d0e70 0x7fef241e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef0406c980 0x7fef0406ee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b5d40 0x7fef24156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 --2- 172.21.15.106:0/2269670582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef240b5660 0x7fef24142460 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 >> 172.21.15.106:0/2269670582 conn(0x7fef2406c740 msgr2=0x7fef240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 shutdown_connections 2024-04-26T21:47:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.547+0000 7fef2b200640 1 -- 172.21.15.106:0/2269670582 wait complete. 2024-04-26T21:47:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:12.735544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:13.898739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:38.567 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:38.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.768+0000 7f7c04c00640 1 Processor -- start 2024-04-26T21:47:38.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.768+0000 7f7c04c00640 1 -- start start 2024-04-26T21:47:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.770+0000 7f7c04c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c000b5ca0 0x7f7c000b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.770+0000 7f7c04c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c000af800 con 0x7f7c0006ce80 2024-04-26T21:47:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.770+0000 7f7c04c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c000ae340 con 0x7f7c0015d100 2024-04-26T21:47:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.770+0000 7f7c04c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7c000afaa0 con 0x7f7c000b5ca0 2024-04-26T21:47:38.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.770+0000 7f7bfd400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7c0006ce80 0x7f7c000b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35618/0 (socket says 172.21.15.106:35618) 2024-04-26T21:47:38.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfd400640 1 -- 172.21.15.106:0/1213813427 learned_addr learned my addr 172.21.15.106:0/1213813427 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:38.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651940208 0 0) 0x7f7c000af800 con 0x7f7c0006ce80 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7be4003520 con 0x7f7c0006ce80 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfe800640 1 --2- 172.21.15.106:0/1213813427 >> v2:172.21.15.204:3300/0 conn(0x7f7c000b5ca0 0x7f7c000b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 521414081 0 0) 0x7f7be4003520 con 0x7f7c0006ce80 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 >> v2:172.21.15.204:3300/0 conn(0x7f7c000b5ca0 msgr2=0x7f7c000b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 --2- 172.21.15.106:0/1213813427 >> v2:172.21.15.204:3300/0 conn(0x7f7c000b5ca0 0x7f7c000b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 >> v1:172.21.15.204:6789/0 conn(0x7f7c0015d100 legacy=0x7f7c0006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.771+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c000afd80 con 0x7f7c0006ce80 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f7bec001670 con 0x7f7c0006ce80 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 >> v1:172.21.15.106:6789/0 conn(0x7f7c0006ce80 legacy=0x7f7c000b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7be4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7be4004800 0x7f7be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be40052e0 0x7f7be400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c000afaa0 con 0x7f7be40039b0 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7be400c1b0 con 0x7f7be4004800 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7be400c3b0 con 0x7f7be40052e0 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7be4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 -- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be40052e0 msgr2=0x7f7be400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be40052e0 0x7f7be400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 -- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7be4004800 msgr2=0x7f7be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7be4004800 0x7f7be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfde00640 1 -- 172.21.15.106:0/1213813427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be4003520 con 0x7f7be40039b0 2024-04-26T21:47:38.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.772+0000 7f7bfe800640 1 -- 172.21.15.106:0/1213813427 reap_dead start 2024-04-26T21:47:38.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.773+0000 7f7bfde00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7be4003dd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0001720 tx=0x7f7be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26706dcda4b090a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0001370 con 0x7f7be40039b0 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf00052d0 con 0x7f7be40039b0 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7bfca00640 1 -- 172.21.15.106:0/1213813427 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002ab90 con 0x7f7be40039b0 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 -- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 msgr2=0x7f7be4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7be4003dd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0001720 tx=0x7f7be4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 -- 172.21.15.106:0/1213813427 shutdown_connections 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 --2- 172.21.15.106:0/1213813427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7be4003dd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 -- 172.21.15.106:0/1213813427 >> 172.21.15.106:0/1213813427 conn(0x7f7c000b2d50 msgr2=0x7f7c000b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 -- 172.21.15.106:0/1213813427 shutdown_connections 2024-04-26T21:47:38.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.774+0000 7f7c04c00640 1 -- 172.21.15.106:0/1213813427 wait complete. 2024-04-26T21:47:38.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.776+0000 7f7c04c00640 1 Processor -- start 2024-04-26T21:47:38.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.776+0000 7f7c04c00640 1 -- start start 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c00146c70 0x7f7c0009cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c00159eb0 0x7f7c0015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0009ee50 con 0x7f7be40039b0 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c000b0340 con 0x7f7c00159eb0 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.777+0000 7f7c04c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c00160b70 con 0x7f7c00146c70 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55010/0 (socket says 172.21.15.106:55010) 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 -- 172.21.15.106:0/127475739 learned_addr learned my addr 172.21.15.106:0/127475739 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 -- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c00146c70 msgr2=0x7f7c0009cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c00146c70 0x7f7c0009cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 -- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c00159eb0 msgr2=0x7f7c0015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c00159eb0 0x7f7c0015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 -- 172.21.15.106:0/127475739 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0015e4f0 con 0x7f7be40039b0 2024-04-26T21:47:38.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.778+0000 7f7bfde00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0001840 tx=0x7f7bf0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.779+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf003f020 con 0x7f7be40039b0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.779+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf0005480 con 0x7f7be40039b0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.779+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0015d730 con 0x7f7be40039b0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.779+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0007a30 con 0x7f7be40039b0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.779+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0015dcf0 con 0x7f7be40039b0 2024-04-26T21:47:38.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.780+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c00069650 con 0x7f7be40039b0 2024-04-26T21:47:38.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.781+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7bf0042020 con 0x7f7be40039b0 2024-04-26T21:47:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.782+0000 7f7bf6000640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 0x7f7be006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.782+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7bf00ab750 con 0x7f7be40039b0 2024-04-26T21:47:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.782+0000 7f7bfd400640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 0x7f7be006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.783+0000 7f7bfd400640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 0x7f7be006ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0011e0 tx=0x7f7bec001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.786+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bf0049030 con 0x7f7be40039b0 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.983+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c00073d00 con 0x7f7be40039b0 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.984+0000 7f7bf6000640 1 -- 172.21.15.106:0/127475739 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f7bf003a360 con 0x7f7be40039b0 2024-04-26T21:47:38.986 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 msgr2=0x7f7be006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 0x7f7be006ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0011e0 tx=0x7f7bec001520 comp rx=0 tx=0).stop 2024-04-26T21:47:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 msgr2=0x7f7c00159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0001840 tx=0x7f7bf0001120 comp rx=0 tx=0).stop 2024-04-26T21:47:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 shutdown_connections 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c00159eb0 0x7f7c0015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7be006c980 0x7f7be006ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c00146c70 0x7f7c0009cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 --2- 172.21.15.106:0/127475739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be40039b0 0x7f7c00159270 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 >> 172.21.15.106:0/127475739 conn(0x7f7c000b2d50 msgr2=0x7f7c000b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 shutdown_connections 2024-04-26T21:47:38.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:38.987+0000 7f7c04c00640 1 -- 172.21.15.106:0/127475739 wait complete. 2024-04-26T21:47:39.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.207+0000 7f1e74600640 1 Processor -- start 2024-04-26T21:47:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.207+0000 7f1e74600640 1 -- start start 2024-04-26T21:47:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.208+0000 7f1e74600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e6c0d19c0 0x7f1e6c0d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e74600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e74600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e74600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c0d40d0 con 0x7f1e6c0b6170 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e74600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c0d3830 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e74600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d3b10 con 0x7f1e6c0d1da0 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e71200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e71200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55022/0 (socket says 172.21.15.106:55022) 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e71200640 1 -- 172.21.15.106:0/855545630 learned_addr learned my addr 172.21.15.106:0/855545630 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e71c00640 1 --2- 172.21.15.106:0/855545630 >> v2:172.21.15.204:3300/0 conn(0x7f1e6c0d19c0 0x7f1e6c0d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.209+0000 7f1e72600640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.210+0000 7f1e71200640 1 -- 172.21.15.106:0/855545630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 msgr2=0x7f1e6c162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.210+0000 7f1e71200640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.210+0000 7f1e71200640 1 -- 172.21.15.106:0/855545630 >> v2:172.21.15.204:3300/0 conn(0x7f1e6c0d19c0 msgr2=0x7f1e6c0d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.210+0000 7f1e71200640 1 --2- 172.21.15.106:0/855545630 >> v2:172.21.15.204:3300/0 conn(0x7f1e6c0d19c0 0x7f1e6c0d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.210+0000 7f1e71200640 1 -- 172.21.15.106:0/855545630 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0d3df0 con 0x7f1e6c0b6170 2024-04-26T21:47:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.211+0000 7f1e71200640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c06ca80 tx=0x7f1e5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d32214f17b5caa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.211+0000 7f1e6be00640 1 -- 172.21.15.106:0/855545630 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58037070 con 0x7f1e6c0b6170 2024-04-26T21:47:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.211+0000 7f1e6be00640 1 -- 172.21.15.106:0/855545630 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e58032440 con 0x7f1e6c0b6170 2024-04-26T21:47:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.211+0000 7f1e6be00640 1 -- 172.21.15.106:0/855545630 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58032810 con 0x7f1e6c0b6170 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 -- 172.21.15.106:0/855545630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0b6170 msgr2=0x7f1e6c0c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c06ca80 tx=0x7f1e5802a140 comp rx=0 tx=0).stop 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 -- 172.21.15.106:0/855545630 shutdown_connections 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 --2- 172.21.15.106:0/855545630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8c70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 --2- 172.21.15.106:0/855545630 >> v2:172.21.15.204:3300/0 conn(0x7f1e6c0d19c0 0x7f1e6c0d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 -- 172.21.15.106:0/855545630 >> 172.21.15.106:0/855545630 conn(0x7f1e6c09e460 msgr2=0x7f1e6c0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 -- 172.21.15.106:0/855545630 shutdown_connections 2024-04-26T21:47:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.212+0000 7f1e74600640 1 -- 172.21.15.106:0/855545630 wait complete. 2024-04-26T21:47:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.213+0000 7f1e74600640 1 Processor -- start 2024-04-26T21:47:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.214+0000 7f1e74600640 1 -- start start 2024-04-26T21:47:39.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c0a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0cfc30 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0c91b0 con 0x7f1e6c0b6170 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e74600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c162b80 con 0x7f1e6c0d1da0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55036/0 (socket says 172.21.15.106:55036) 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 -- 172.21.15.106:0/4026683784 learned_addr learned my addr 172.21.15.106:0/4026683784 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71c00640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 -- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 msgr2=0x7f1e6c0a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c0a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 -- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e6c0b6170 msgr2=0x7f1e6c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.215+0000 7f1e71200640 1 -- 172.21.15.106:0/4026683784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0c8d50 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e71200640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000c00 tx=0x7f1e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58041020 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e580306c0 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58037040 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c0b47a0 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.216+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c0b4a40 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.217+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c0c9a20 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.218+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1e58044020 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.219+0000 7f1e69600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 0x7f1e4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.219+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1e580ac4f0 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.219+0000 7f1e71c00640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 0x7f1e4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.220+0000 7f1e71c00640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 0x7f1e4c06ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.224+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e6c0c9a20 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.412+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e6c0d26d0 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.413+0000 7f1e69600640 1 -- 172.21.15.106:0/4026683784 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f1e5804b030 con 0x7f1e6c0d19c0 2024-04-26T21:47:39.414 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 msgr2=0x7f1e4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 0x7f1e4c06ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 msgr2=0x7f1e6c1f5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000c00 tx=0x7f1e580046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 shutdown_connections 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1e4c06c980 0x7f1e4c06ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e6c0d1da0 0x7f1e6c0a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e6c0d19c0 0x7f1e6c1f5f70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 --2- 172.21.15.106:0/4026683784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e6c0b6170 0x7f1e6c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.415+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 >> 172.21.15.106:0/4026683784 conn(0x7f1e6c09e460 msgr2=0x7f1e6c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.416+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 shutdown_connections 2024-04-26T21:47:39.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.416+0000 7f1e74600640 1 -- 172.21.15.106:0/4026683784 wait complete. 2024-04-26T21:47:39.433 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:39.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.633+0000 7f3285200640 1 Processor -- start 2024-04-26T21:47:39.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.633+0000 7f3285200640 1 -- start start 2024-04-26T21:47:39.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.634+0000 7f3285200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f3285200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32800d34e0 0x7f32800d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f3285200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 0x7f32800cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f3285200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f32800d3220 con 0x7f32800cfb10 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f3285200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f32800b8b60 con 0x7f32800d34e0 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f3285200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f32800b8e40 con 0x7f328015f330 2024-04-26T21:47:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f327fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 0x7f32800cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f327fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 0x7f32800cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55050/0 (socket says 172.21.15.106:55050) 2024-04-26T21:47:39.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.635+0000 7f327fe00640 1 -- 172.21.15.106:0/3000887549 learned_addr learned my addr 172.21.15.106:0/3000887549 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:39.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 --2- 172.21.15.106:0/3000887549 >> v2:172.21.15.204:3300/0 conn(0x7f32800d34e0 0x7f32800d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f327f400640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 -- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 msgr2=0x7f32800d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 -- 172.21.15.106:0/3000887549 >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 msgr2=0x7f32800cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 --2- 172.21.15.106:0/3000887549 >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 0x7f32800cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.636+0000 7f3277e00640 1 -- 172.21.15.106:0/3000887549 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800b9120 con 0x7f32800d34e0 2024-04-26T21:47:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.637+0000 7f327f400640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.638+0000 7f3277e00640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32800d3960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f328008d470 tx=0x7f326802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae2c023fdde22659 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f327ea00640 1 -- 172.21.15.106:0/3000887549 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268037070 con 0x7f32800d34e0 2024-04-26T21:47:39.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f327ea00640 1 -- 172.21.15.106:0/3000887549 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3268032440 con 0x7f32800d34e0 2024-04-26T21:47:39.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f327ea00640 1 -- 172.21.15.106:0/3000887549 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326802ae60 con 0x7f32800d34e0 2024-04-26T21:47:39.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 -- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 msgr2=0x7f32800d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32800d3960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f328008d470 tx=0x7f326802a110 comp rx=0 tx=0).stop 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 -- 172.21.15.106:0/3000887549 shutdown_connections 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 --2- 172.21.15.106:0/3000887549 >> v2:172.21.15.106:3300/0 conn(0x7f32800cfb10 0x7f32800cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32800d3960 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.644+0000 7f3285200640 1 --2- 172.21.15.106:0/3000887549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.645+0000 7f3285200640 1 -- 172.21.15.106:0/3000887549 >> 172.21.15.106:0/3000887549 conn(0x7f328009e320 msgr2=0x7f32800d4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.645+0000 7f3285200640 1 -- 172.21.15.106:0/3000887549 shutdown_connections 2024-04-26T21:47:39.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.645+0000 7f3285200640 1 -- 172.21.15.106:0/3000887549 wait complete. 2024-04-26T21:47:39.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.647+0000 7f3285200640 1 Processor -- start 2024-04-26T21:47:39.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.647+0000 7f3285200640 1 -- start start 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32801622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32800d5cc0 con 0x7f32800cfb10 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32800b96e0 con 0x7f32800d34e0 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f3285200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f32800b87a0 con 0x7f328015f330 2024-04-26T21:47:39.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f327f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f327f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55052/0 (socket says 172.21.15.106:55052) 2024-04-26T21:47:39.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.648+0000 7f327f400640 1 -- 172.21.15.106:0/2241911310 learned_addr learned my addr 172.21.15.106:0/2241911310 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 -- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 msgr2=0x7f32800ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 -- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 msgr2=0x7f32801622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32801622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 -- 172.21.15.106:0/2241911310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800cd7d0 con 0x7f32800cfb10 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f327f400640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f327000ec10 tx=0x7f327000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270017070 con 0x7f32800cfb10 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327000cbb0 con 0x7f32800cfb10 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32800cdab0 con 0x7f32800cfb10 2024-04-26T21:47:39.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270013470 con 0x7f32800cfb10 2024-04-26T21:47:39.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.649+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32800ce070 con 0x7f32800cfb10 2024-04-26T21:47:39.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.651+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32800693c0 con 0x7f32800cfb10 2024-04-26T21:47:39.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.651+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f327001a070 con 0x7f32800cfb10 2024-04-26T21:47:39.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.652+0000 7f3277400640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 0x7f325c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.652+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f327008c140 con 0x7f32800cfb10 2024-04-26T21:47:39.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.653+0000 7f3277e00640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 0x7f325c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.653+0000 7f3277e00640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 0x7f325c06ef90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3268034000 tx=0x7f326802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.657+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f327005a2f0 con 0x7f32800cfb10 2024-04-26T21:47:39.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.857+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32800cbc70 con 0x7f32800cfb10 2024-04-26T21:47:39.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.858+0000 7f3277400640 1 -- 172.21.15.106:0/2241911310 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f3270059bf0 con 0x7f32800cfb10 2024-04-26T21:47:39.859 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 msgr2=0x7f325c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 0x7f325c06ef90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3268034000 tx=0x7f326802c000 comp rx=0 tx=0).stop 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 msgr2=0x7f3280151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f327000ec10 tx=0x7f327000c520 comp rx=0 tx=0).stop 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 shutdown_connections 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f325c06cad0 0x7f325c06ef90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328015f330 0x7f32800ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32800d34e0 0x7f32801622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 --2- 172.21.15.106:0/2241911310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32800cfb10 0x7f3280151d80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 >> 172.21.15.106:0/2241911310 conn(0x7f328009e320 msgr2=0x7f32800cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 shutdown_connections 2024-04-26T21:47:39.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:39.861+0000 7f3285200640 1 -- 172.21.15.106:0/2241911310 wait complete. 2024-04-26T21:47:39.881 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:47:40.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.075+0000 7f1b54a00640 1 Processor -- start 2024-04-26T21:47:40.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.075+0000 7f1b54a00640 1 -- start start 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 0x7f1b500b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b54a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b50146a60 con 0x7f1b50146680 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b54a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5006dd60 con 0x7f1b500d0e70 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b54a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5006e970 con 0x7f1b500d12b0 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b4f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1b50146680 0x7f1b500b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35634/0 (socket says 172.21.15.106:35634) 2024-04-26T21:47:40.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.077+0000 7f1b4f400640 1 -- 172.21.15.106:0/1869924559 learned_addr learned my addr 172.21.15.106:0/1869924559 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4fe00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 0x7f1b500b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2862805266 0 0) 0x7f1b50146a60 con 0x7f1b50146680 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b34003520 con 0x7f1b50146680 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2686062235 0 0) 0x7f1b34003520 con 0x7f1b50146680 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 msgr2=0x7f1b500b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 0x7f1b500b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 >> v1:172.21.15.204:6789/0 conn(0x7f1b500d0e70 legacy=0x7f1b500b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5006e040 con 0x7f1b50146680 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f1b40001670 con 0x7f1b50146680 2024-04-26T21:47:40.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 >> v1:172.21.15.106:6789/0 conn(0x7f1b50146680 legacy=0x7f1b500b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34004800 0x7f1b34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 0x7f1b34005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3400bb50 con 0x7f1b340039e0 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3400bcd0 con 0x7f1b34004800 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.078+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3400bed0 con 0x7f1b500d12b0 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 -- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 msgr2=0x7f1b34005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b500d12b0 0x7f1b34005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 -- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34004800 msgr2=0x7f1b34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34004800 0x7f1b34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.079+0000 7f1b4f400640 1 -- 172.21.15.106:0/1869924559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b34003520 con 0x7f1b340039e0 2024-04-26T21:47:40.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.080+0000 7f1b4f400640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b34003e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c190 tx=0x7f1b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed34f17aaa9007 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.080+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007c60 con 0x7f1b340039e0 2024-04-26T21:47:40.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.080+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40005ad0 con 0x7f1b340039e0 2024-04-26T21:47:40.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.080+0000 7f1b4ea00640 1 -- 172.21.15.106:0/1869924559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007070 con 0x7f1b340039e0 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 -- 172.21.15.106:0/1869924559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 msgr2=0x7f1b34003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 --2- 172.21.15.106:0/1869924559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b34003e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c190 tx=0x7f1b34005830 comp rx=0 tx=0).stop 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b4fe00640 1 -- 172.21.15.106:0/1869924559 reap_dead start 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 -- 172.21.15.106:0/1869924559 shutdown_connections 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 -- 172.21.15.106:0/1869924559 >> 172.21.15.106:0/1869924559 conn(0x7f1b5006c740 msgr2=0x7f1b5006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 -- 172.21.15.106:0/1869924559 shutdown_connections 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.081+0000 7f1b54a00640 1 -- 172.21.15.106:0/1869924559 wait complete. 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.083+0000 7f1b54a00640 1 Processor -- start 2024-04-26T21:47:40.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.083+0000 7f1b54a00640 1 -- start start 2024-04-26T21:47:40.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b50073760 0x7f1b50072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b5015c530 0x7f1b500b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5006e600 con 0x7f1b340039e0 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b500b60a0 con 0x7f1b50073760 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b54a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b500d1740 con 0x7f1b5015c530 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b4f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b4f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55070/0 (socket says 172.21.15.106:55070) 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b4f400640 1 -- 172.21.15.106:0/655252909 learned_addr learned my addr 172.21.15.106:0/655252909 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.084+0000 7f1b4fe00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b5015c530 0x7f1b500b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 -- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b5015c530 msgr2=0x7f1b500b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b5015c530 0x7f1b500b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 -- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b50073760 msgr2=0x7f1b50072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b47e00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b50073760 0x7f1b50072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b50073760 0x7f1b50072ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 -- 172.21.15.106:0/655252909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b50073c20 con 0x7f1b340039e0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b4f400640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c2b0 tx=0x7f1b40001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40040020 con 0x7f1b340039e0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b400013b0 con 0x7f1b340039e0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40037040 con 0x7f1b340039e0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b50071fe0 con 0x7f1b340039e0 2024-04-26T21:47:40.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.085+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b500725a0 con 0x7f1b340039e0 2024-04-26T21:47:40.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.087+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b500693c0 con 0x7f1b340039e0 2024-04-26T21:47:40.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.087+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1b40043020 con 0x7f1b340039e0 2024-04-26T21:47:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.089+0000 7f1b47400640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 0x7f1b2c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.089+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b400aba10 con 0x7f1b340039e0 2024-04-26T21:47:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.089+0000 7f1b47e00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 0x7f1b2c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.090+0000 7f1b47e00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 0x7f1b2c06ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b500b4e00 tx=0x7f1b38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.093+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b40005aa0 con 0x7f1b340039e0 2024-04-26T21:47:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.308+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b500b39e0 con 0x7f1b340039e0 2024-04-26T21:47:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.309+0000 7f1b47400640 1 -- 172.21.15.106:0/655252909 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f1b40079b30 con 0x7f1b340039e0 2024-04-26T21:47:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:47:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.311+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 msgr2=0x7f1b2c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.311+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 0x7f1b2c06ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b500b4e00 tx=0x7f1b38009040 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.311+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 msgr2=0x7f1b50160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.311+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c2b0 tx=0x7f1b40001970 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.311+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 shutdown_connections 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1b2c06caa0 0x7f1b2c06ef60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b5015c530 0x7f1b500b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b50073760 0x7f1b50072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 --2- 172.21.15.106:0/655252909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b340039e0 0x7f1b50160ea0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 >> 172.21.15.106:0/655252909 conn(0x7f1b5006c740 msgr2=0x7f1b5006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 shutdown_connections 2024-04-26T21:47:40.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.312+0000 7f1b54a00640 1 -- 172.21.15.106:0/655252909 wait complete. 2024-04-26T21:47:40.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-26T21:47:40.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:40.332 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:40.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[47428]: fuse finished with error 0 and tester_r 0 2024-04-26T21:47:40.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:47:40.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:40.417 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:47:40.618 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:40.618 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:47:40.618 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:40.670 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:47:40.670 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:40.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.975+0000 7f25eda00640 1 Processor -- start 2024-04-26T21:47:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.975+0000 7f25eda00640 1 -- start start 2024-04-26T21:47:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25eda00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e806c770 0x7f25e806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25eda00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 0x7f25e8073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25eda00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e806d090 con 0x7f25e815cfd0 2024-04-26T21:47:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25eda00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f25e80af460 con 0x7f25e806c770 2024-04-26T21:47:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25eda00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f25e80ae850 con 0x7f25e815cb70 2024-04-26T21:47:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25e6a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e806c770 0x7f25e806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25e7400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25e815cfd0 0x7f25e8073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35636/0 (socket says 172.21.15.106:35636) 2024-04-26T21:47:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.977+0000 7f25e7400640 1 -- 172.21.15.106:0/3522645738 learned_addr learned my addr 172.21.15.106:0/3522645738 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25dfe00640 1 --2- 172.21.15.106:0/3522645738 >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 0x7f25e8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25e6a00640 1 -- 172.21.15.106:0/3522645738 >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 msgr2=0x7f25e8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25e6a00640 1 --2- 172.21.15.106:0/3522645738 >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 0x7f25e8073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25e6a00640 1 -- 172.21.15.106:0/3522645738 >> v1:172.21.15.106:6789/0 conn(0x7f25e815cfd0 legacy=0x7f25e8073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25e6a00640 1 -- 172.21.15.106:0/3522645738 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e80aeb30 con 0x7f25e806c770 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25e6000640 1 -- 172.21.15.106:0/3522645738 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584273865 0 0) 0x7f25e806d090 con 0x7f25e815cfd0 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.978+0000 7f25dfe00640 1 --2- 172.21.15.106:0/3522645738 >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 0x7f25e8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.979+0000 7f25e6a00640 1 --2- 172.21.15.106:0/3522645738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e806cb50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f25e80afec0 tx=0x7f25d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4619299f2424d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.979+0000 7f25e6000640 1 -- 172.21.15.106:0/3522645738 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25e806c770 2024-04-26T21:47:40.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.979+0000 7f25e6000640 1 -- 172.21.15.106:0/3522645738 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8032440 con 0x7f25e806c770 2024-04-26T21:47:40.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.979+0000 7f25e6000640 1 -- 172.21.15.106:0/3522645738 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8032810 con 0x7f25e806c770 2024-04-26T21:47:40.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 -- 172.21.15.106:0/3522645738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 msgr2=0x7f25e806cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 --2- 172.21.15.106:0/3522645738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e806cb50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f25e80afec0 tx=0x7f25d802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 -- 172.21.15.106:0/3522645738 shutdown_connections 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 --2- 172.21.15.106:0/3522645738 >> v2:172.21.15.204:3300/0 conn(0x7f25e815cb70 0x7f25e8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 --2- 172.21.15.106:0/3522645738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e806cb50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 -- 172.21.15.106:0/3522645738 >> 172.21.15.106:0/3522645738 conn(0x7f25e80b2d50 msgr2=0x7f25e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 -- 172.21.15.106:0/3522645738 shutdown_connections 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.980+0000 7f25eda00640 1 -- 172.21.15.106:0/3522645738 wait complete. 2024-04-26T21:47:40.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.982+0000 7f25eda00640 1 Processor -- start 2024-04-26T21:47:40.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.982+0000 7f25eda00640 1 -- start start 2024-04-26T21:47:40.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 0x7f25e809c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 0x7f25e814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25e815c600 con 0x7f25e815cb70 2024-04-26T21:47:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25e80af0f0 con 0x7f25e815cfd0 2024-04-26T21:47:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25eda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25e815c780 con 0x7f25e806c770 2024-04-26T21:47:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55098/0 (socket says 172.21.15.106:55098) 2024-04-26T21:47:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.983+0000 7f25dfe00640 1 -- 172.21.15.106:0/1135572307 learned_addr learned my addr 172.21.15.106:0/1135572307 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25e7400640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 0x7f25e814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25e6a00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 0x7f25e809c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 -- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 msgr2=0x7f25e809c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 0x7f25e809c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 -- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 msgr2=0x7f25e814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 0x7f25e814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 -- 172.21.15.106:0/1135572307 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e809e8b0 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25e7400640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 0x7f25e814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25e6a00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 0x7f25e809c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dfe00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c7f0 tx=0x7f25d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0015070 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d0007a30 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0007e00 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.984+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e809cad0 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.985+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e8153680 con 0x7f25e815cb70 2024-04-26T21:47:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.986+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e8153880 con 0x7f25e815cb70 2024-04-26T21:47:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.987+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f25d0018070 con 0x7f25e815cb70 2024-04-26T21:47:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.987+0000 7f25dea00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 0x7f25c406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.988+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f25d008bf30 con 0x7f25e815cb70 2024-04-26T21:47:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.988+0000 7f25e6a00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 0x7f25c406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.988+0000 7f25e6a00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 0x7f25c406ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25d8034000 tx=0x7f25d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:40.992+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25e8153880 con 0x7f25e815cb70 2024-04-26T21:47:41.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.193+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25e8069540 con 0x7f25e815cb70 2024-04-26T21:47:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.195+0000 7f25dea00640 1 -- 172.21.15.106:0/1135572307 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f25d0059fc0 con 0x7f25e815cb70 2024-04-26T21:47:41.196 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:41.196 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 msgr2=0x7f25c406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 0x7f25c406ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25d8034000 tx=0x7f25d802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 msgr2=0x7f25e809e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c7f0 tx=0x7f25d000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 shutdown_connections 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cfd0 0x7f25e814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e809e370 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c406caa0 0x7f25c406ef60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 --2- 172.21.15.106:0/1135572307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e806c770 0x7f25e809c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 >> 172.21.15.106:0/1135572307 conn(0x7f25e80b2d50 msgr2=0x7f25e815f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 shutdown_connections 2024-04-26T21:47:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.198+0000 7f25eda00640 1 -- 172.21.15.106:0/1135572307 wait complete. 2024-04-26T21:47:41.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:12.735544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:13.898739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:41.219 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.424+0000 7ff76ae00640 1 Processor -- start 2024-04-26T21:47:41.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.424+0000 7ff76ae00640 1 -- start start 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff76ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff7641466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff76ae00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7640b4d20 con 0x7ff764160d60 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff76ae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7640b5000 con 0x7ff764161140 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff76ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff76406e5f0 con 0x7ff764146280 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff769a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff764160d60 0x7ff7640b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35648/0 (socket says 172.21.15.106:35648) 2024-04-26T21:47:41.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.426+0000 7ff769a00640 1 -- 172.21.15.106:0/3071184660 learned_addr learned my addr 172.21.15.106:0/3071184660 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422452465 0 0) 0x7ff7640b4d20 con 0x7ff764160d60 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff74c003520 con 0x7ff764160d60 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff76a400640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff7641466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2060742293 0 0) 0x7ff74c003520 con 0x7ff764160d60 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 msgr2=0x7ff7641466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff7641466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 >> v1:172.21.15.204:6789/0 conn(0x7ff764161140 legacy=0x7ff76406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640b52e0 con 0x7ff764160d60 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7ff758001670 con 0x7ff764160d60 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 >> v1:172.21.15.106:6789/0 conn(0x7ff764160d60 legacy=0x7ff7640b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff74c003a50 0x7ff74c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff74c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c004d90 0x7ff74c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff76406e5f0 con 0x7ff74c003a50 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c00bb80 con 0x7ff74c004d90 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c00bd80 con 0x7ff764146280 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.427+0000 7ff769a00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff74c003a50 0x7ff74c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769000640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff74c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff76a400640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c004d90 0x7ff74c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769a00640 1 -- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 msgr2=0x7ff74c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769a00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff74c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769a00640 1 -- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c004d90 msgr2=0x7ff74c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769a00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c004d90 0x7ff74c005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769a00640 1 -- 172.21.15.106:0/3071184660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c003520 con 0x7ff74c003a50 2024-04-26T21:47:41.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.428+0000 7ff769000640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff764146280 0x7ff74c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.429+0000 7ff769a00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff74c003a50 0x7ff74c003e70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c240 tx=0x7ff74c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a751ad49a94b6c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.429+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758007cf0 con 0x7ff74c003a50 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.429+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7580058d0 con 0x7ff74c003a50 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.429+0000 7ff763e00640 1 -- 172.21.15.106:0/3071184660 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758005ce0 con 0x7ff74c003a50 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 -- 172.21.15.106:0/3071184660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff74c003a50 msgr2=0x7ff74c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 --2- 172.21.15.106:0/3071184660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff74c003a50 0x7ff74c003e70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c240 tx=0x7ff74c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76a400640 1 -- 172.21.15.106:0/3071184660 reap_dead start 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 -- 172.21.15.106:0/3071184660 shutdown_connections 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 -- 172.21.15.106:0/3071184660 >> 172.21.15.106:0/3071184660 conn(0x7ff7640b2900 msgr2=0x7ff7640b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 -- 172.21.15.106:0/3071184660 shutdown_connections 2024-04-26T21:47:41.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.430+0000 7ff76ae00640 1 -- 172.21.15.106:0/3071184660 wait complete. 2024-04-26T21:47:41.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.432+0000 7ff76ae00640 1 Processor -- start 2024-04-26T21:47:41.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.432+0000 7ff76ae00640 1 -- start start 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c003a50 0x7ff7641f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff76415a610 0x7ff764140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640cf6f0 con 0x7ff76413f620 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff764146c30 con 0x7ff74c003a50 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff764161770 con 0x7ff76415a610 2024-04-26T21:47:41.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55116/0 (socket says 172.21.15.106:55116) 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 -- 172.21.15.106:0/2064309607 learned_addr learned my addr 172.21.15.106:0/2064309607 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff76a400640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff76415a610 0x7ff764140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769a00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c003a50 0x7ff7641f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 -- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff76415a610 msgr2=0x7ff764140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff76415a610 0x7ff764140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 -- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c003a50 msgr2=0x7ff7641f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.433+0000 7ff769000640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c003a50 0x7ff7641f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff769000640 1 -- 172.21.15.106:0/2064309607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76415c8a0 con 0x7ff76413f620 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff769000640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff75000d7a0 tx=0x7ff75000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750005070 con 0x7ff76413f620 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff750005210 con 0x7ff76413f620 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750007030 con 0x7ff76413f620 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76415cb80 con 0x7ff76413f620 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.434+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641f2b60 con 0x7ff76413f620 2024-04-26T21:47:41.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.435+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7640693c0 con 0x7ff76413f620 2024-04-26T21:47:41.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.436+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff75001a020 con 0x7ff76413f620 2024-04-26T21:47:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.437+0000 7ff761600640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 0x7ff74406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.437+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff75008b8d0 con 0x7ff76413f620 2024-04-26T21:47:41.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.438+0000 7ff769a00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 0x7ff74406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.438+0000 7ff769a00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 0x7ff74406ef60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff758033000 tx=0x7ff758009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.442+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7500598d0 con 0x7ff76413f620 2024-04-26T21:47:41.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.639+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7640b5f20 con 0x7ff76413f620 2024-04-26T21:47:41.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.643+0000 7ff761600640 1 -- 172.21.15.106:0/2064309607 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7ff7500591d0 con 0x7ff76413f620 2024-04-26T21:47:41.645 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:41.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 msgr2=0x7ff74406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 0x7ff74406ef60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff758033000 tx=0x7ff758009030 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 msgr2=0x7ff76415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff75000d7a0 tx=0x7ff75000dc60 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 shutdown_connections 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff74406caa0 0x7ff74406ef60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff76415a610 0x7ff764140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff76413f620 0x7ff76415bf30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 --2- 172.21.15.106:0/2064309607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c003a50 0x7ff7641f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.646+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 >> 172.21.15.106:0/2064309607 conn(0x7ff7640b2900 msgr2=0x7ff7640d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.647+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 shutdown_connections 2024-04-26T21:47:41.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.647+0000 7ff76ae00640 1 -- 172.21.15.106:0/2064309607 wait complete. 2024-04-26T21:47:41.667 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:41.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.849+0000 7fa6e5800640 1 Processor -- start 2024-04-26T21:47:41.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.849+0000 7fa6e5800640 1 -- start start 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6e5800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa6e006ce80 0x7fa6e00b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6e00b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6e5800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e00af800 con 0x7fa6e015d100 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6e5800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e00ae340 con 0x7fa6e006ce80 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6e5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00afaa0 con 0x7fa6e00b5ca0 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6dea00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa6e015d100 0x7fa6e006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35662/0 (socket says 172.21.15.106:35662) 2024-04-26T21:47:41.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6dea00640 1 -- 172.21.15.106:0/3884135311 learned_addr learned my addr 172.21.15.106:0/3884135311 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.851+0000 7fa6de000640 1 --2- 172.21.15.106:0/3884135311 >> v2:172.21.15.204:3300/0 conn(0x7fa6e006ce80 0x7fa6e00b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6df400640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6e00b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848110076 0 0) 0x7fa6e00af800 con 0x7fa6e015d100 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6c4003520 con 0x7fa6e015d100 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 222379197 0 0) 0x7fa6c4003520 con 0x7fa6e015d100 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 msgr2=0x7fa6e00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6e00b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 >> v2:172.21.15.204:3300/0 conn(0x7fa6e006ce80 msgr2=0x7fa6e00b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 --2- 172.21.15.106:0/3884135311 >> v2:172.21.15.204:3300/0 conn(0x7fa6e006ce80 0x7fa6e00b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00afd80 con 0x7fa6e015d100 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fa6d0001670 con 0x7fa6e015d100 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 >> v1:172.21.15.106:6789/0 conn(0x7fa6e015d100 legacy=0x7fa6e006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6c40039b0 0x7fa6c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6c4004800 0x7fa6c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00ae340 con 0x7fa6c40039b0 2024-04-26T21:47:41.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00afaa0 con 0x7fa6c4004800 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.852+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c400b9c0 con 0x7fa6e00b5ca0 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6c40039b0 0x7fa6c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6de000640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6c4004800 0x7fa6c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6df400640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 -- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 msgr2=0x7fa6c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00b5ca0 0x7fa6c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 -- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6c4004800 msgr2=0x7fa6c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6c4004800 0x7fa6c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.853+0000 7fa6dea00640 1 -- 172.21.15.106:0/3884135311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4003520 con 0x7fa6c40039b0 2024-04-26T21:47:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.854+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6c40039b0 0x7fa6c4003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c290 tx=0x7fa6c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4843bac48c27db06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.854+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0007d50 con 0x7fa6c40039b0 2024-04-26T21:47:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.854+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d0005af0 con 0x7fa6c40039b0 2024-04-26T21:47:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.854+0000 7fa6dd600640 1 -- 172.21.15.106:0/3884135311 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0007070 con 0x7fa6c40039b0 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 -- 172.21.15.106:0/3884135311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6c40039b0 msgr2=0x7fa6c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3884135311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6c40039b0 0x7fa6c4003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c290 tx=0x7fa6c4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6df400640 1 -- 172.21.15.106:0/3884135311 reap_dead start 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 -- 172.21.15.106:0/3884135311 shutdown_connections 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 -- 172.21.15.106:0/3884135311 >> 172.21.15.106:0/3884135311 conn(0x7fa6e00b2d50 msgr2=0x7fa6e00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 -- 172.21.15.106:0/3884135311 shutdown_connections 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.855+0000 7fa6e5800640 1 -- 172.21.15.106:0/3884135311 wait complete. 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.857+0000 7fa6e5800640 1 Processor -- start 2024-04-26T21:47:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.858+0000 7fa6e5800640 1 -- start start 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 0x7fa6e0150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 0x7fa6e00b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e009ee50 con 0x7fa6e009dbe0 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00b0340 con 0x7fa6e00b4e50 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6e5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e0160b70 con 0x7fa6c40039b0 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55136/0 (socket says 172.21.15.106:55136) 2024-04-26T21:47:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 -- 172.21.15.106:0/3829351520 learned_addr learned my addr 172.21.15.106:0/3829351520 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:41.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 0x7fa6e0150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6df400640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 0x7fa6e00b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 -- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 msgr2=0x7fa6e0150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 0x7fa6e0150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 -- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 msgr2=0x7fa6e00b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 0x7fa6e00b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6de000640 1 -- 172.21.15.106:0/3829351520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e015e660 con 0x7fa6e009dbe0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6df400640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 0x7fa6e00b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.859+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 0x7fa6e0150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6de000640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc011540 tx=0x7fa6cc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc015930 con 0x7fa6e009dbe0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc015ad0 con 0x7fa6e009dbe0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01d400 con 0x7fa6e009dbe0 2024-04-26T21:47:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e015e940 con 0x7fa6e009dbe0 2024-04-26T21:47:41.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.860+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e015ef00 con 0x7fa6e009dbe0 2024-04-26T21:47:41.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.861+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e0073250 con 0x7fa6e009dbe0 2024-04-26T21:47:41.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.862+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa6cc01d5a0 con 0x7fa6e009dbe0 2024-04-26T21:47:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.863+0000 7fa6d6a00640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 0x7fa6c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.863+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa6cc08c5e0 con 0x7fa6e009dbe0 2024-04-26T21:47:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.864+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 0x7fa6c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.864+0000 7fa6dea00640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 0x7fa6c006ef60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0033000 tx=0x7fa6d0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:41.868+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6e0073250 con 0x7fa6e009dbe0 2024-04-26T21:47:42.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.070+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa6e0074220 con 0x7fa6e009dbe0 2024-04-26T21:47:42.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.071+0000 7fa6d6a00640 1 -- 172.21.15.106:0/3829351520 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fa6e0074220 con 0x7fa6e009dbe0 2024-04-26T21:47:42.072 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 msgr2=0x7fa6c006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 0x7fa6c006ef60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0033000 tx=0x7fa6d0009030 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 msgr2=0x7fa6e015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc011540 tx=0x7fa6cc011a00 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 shutdown_connections 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e00b4e50 0x7fa6e00b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6e009dbe0 0x7fa6e015def0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6c006caa0 0x7fa6c006ef60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 --2- 172.21.15.106:0/3829351520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6c40039b0 0x7fa6e0150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.073+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 >> 172.21.15.106:0/3829351520 conn(0x7fa6e00b2d50 msgr2=0x7fa6e00b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.074+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 shutdown_connections 2024-04-26T21:47:42.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.074+0000 7fa6e5800640 1 -- 172.21.15.106:0/3829351520 wait complete. 2024-04-26T21:47:42.094 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.291+0000 7fe367e00640 1 Processor -- start 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.291+0000 7fe367e00640 1 -- start start 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3681632e0 0x7fe3681636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b6210 0x7fe3680b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3680b6710 con 0x7fe3680d3110 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe3680d3600 con 0x7fe3681632e0 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3680d38e0 con 0x7fe3680b6210 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe366000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe3680d3110 0x7fe368162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35676/0 (socket says 172.21.15.106:35676) 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe366000640 1 -- 172.21.15.106:0/2505298351 learned_addr learned my addr 172.21.15.106:0/2505298351 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe366a00640 1 --2- 172.21.15.106:0/2505298351 >> v2:172.21.15.204:3300/0 conn(0x7fe3681632e0 0x7fe3681636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182626979 0 0) 0x7fe3680b6710 con 0x7fe3680d3110 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b6210 0x7fe3680b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.293+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe34c003520 con 0x7fe3680d3110 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3845166911 0 0) 0x7fe34c003520 con 0x7fe3680d3110 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b6210 msgr2=0x7fe3680b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b6210 0x7fe3680b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 >> v2:172.21.15.204:3300/0 conn(0x7fe3681632e0 msgr2=0x7fe3681636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 --2- 172.21.15.106:0/2505298351 >> v2:172.21.15.204:3300/0 conn(0x7fe3681632e0 0x7fe3681636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3680d3bc0 con 0x7fe3680d3110 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fe354001670 con 0x7fe3680d3110 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 >> v1:172.21.15.106:6789/0 conn(0x7fe3680d3110 legacy=0x7fe368162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe34c0039b0 0x7fe34c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe34c0051c0 0x7fe34c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3680d3600 con 0x7fe34c0051c0 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3680d38e0 con 0x7fe34c0039b0 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe34c00bf80 con 0x7fe34c004800 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b6210 0x7fe3680b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.294+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe34c0051c0 0x7fe34c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe366a00640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe34c0039b0 0x7fe34c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe366000640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe367400640 1 -- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c004800 msgr2=0x7fe34c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe367400640 1 -- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe34c0039b0 msgr2=0x7fe34c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe34c0039b0 0x7fe34c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe367400640 1 -- 172.21.15.106:0/2505298351 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c003520 con 0x7fe34c0051c0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.295+0000 7fe366000640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe367400640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe34c0051c0 0x7fe34c00b9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe36806cbf0 tx=0x7fe34c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fc380f419a97c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360030050 con 0x7fe34c0051c0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe360003b90 con 0x7fe34c0051c0 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe365600640 1 -- 172.21.15.106:0/2505298351 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360009b50 con 0x7fe34c0051c0 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe367e00640 1 -- 172.21.15.106:0/2505298351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe34c0051c0 msgr2=0x7fe34c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe367e00640 1 --2- 172.21.15.106:0/2505298351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe34c0051c0 0x7fe34c00b9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe36806cbf0 tx=0x7fe34c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.296+0000 7fe367400640 1 -- 172.21.15.106:0/2505298351 reap_dead start 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.297+0000 7fe367e00640 1 -- 172.21.15.106:0/2505298351 shutdown_connections 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.297+0000 7fe367e00640 1 -- 172.21.15.106:0/2505298351 >> 172.21.15.106:0/2505298351 conn(0x7fe36809e2f0 msgr2=0x7fe3680d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.297+0000 7fe367e00640 1 -- 172.21.15.106:0/2505298351 shutdown_connections 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.297+0000 7fe367e00640 1 -- 172.21.15.106:0/2505298351 wait complete. 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.298+0000 7fe367e00640 1 Processor -- start 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.299+0000 7fe367e00640 1 -- start start 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.299+0000 7fe367e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c0051c0 0x7fe3680b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.299+0000 7fe367e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.299+0000 7fe367e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36806e1c0 0x7fe36806e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe367e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe368163d80 con 0x7fe3680a2960 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe367e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe368163c00 con 0x7fe36806e1c0 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe367e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe368163f00 con 0x7fe34c0051c0 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55160/0 (socket says 172.21.15.106:55160) 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 -- 172.21.15.106:0/622767212 learned_addr learned my addr 172.21.15.106:0/622767212 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 -- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c0051c0 msgr2=0x7fe3680b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c0051c0 0x7fe3680b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 -- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36806e1c0 msgr2=0x7fe36806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe367400640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36806e1c0 0x7fe36806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36806e1c0 0x7fe36806e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.300+0000 7fe366000640 1 -- 172.21.15.106:0/622767212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3680b48f0 con 0x7fe3680a2960 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe366000640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe35400d7e0 tx=0x7fe35400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354019070 con 0x7fe3680a2960 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3540074e0 con 0x7fe3680a2960 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3680b4bd0 con 0x7fe3680a2960 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354004a20 con 0x7fe3680a2960 2024-04-26T21:47:42.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.301+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3680b5190 con 0x7fe3680a2960 2024-04-26T21:47:42.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.302+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3680cf3c0 con 0x7fe3680a2960 2024-04-26T21:47:42.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.303+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe35401c070 con 0x7fe3680a2960 2024-04-26T21:47:42.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.304+0000 7fe35ea00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 0x7fe34806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.304+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe35408bab0 con 0x7fe3680a2960 2024-04-26T21:47:42.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.305+0000 7fe366a00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 0x7fe34806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.305+0000 7fe366a00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 0x7fe34806ef90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe358009910 tx=0x7fe358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.309+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3680cf3c0 con 0x7fe3680a2960 2024-04-26T21:47:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.493+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe3680693c0 con 0x7fe3680a2960 2024-04-26T21:47:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.494+0000 7fe35ea00640 1 -- 172.21.15.106:0/622767212 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fe354059b40 con 0x7fe3680a2960 2024-04-26T21:47:42.495 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:47:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 msgr2=0x7fe34806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 0x7fe34806ef90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe358009910 tx=0x7fe358008040 comp rx=0 tx=0).stop 2024-04-26T21:47:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 msgr2=0x7fe3681534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe35400d7e0 tx=0x7fe35400dca0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 shutdown_connections 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36806e1c0 0x7fe36806e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3680a2960 0x7fe3681534e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fe34806cad0 0x7fe34806ef90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 --2- 172.21.15.106:0/622767212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe34c0051c0 0x7fe3680b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 >> 172.21.15.106:0/622767212 conn(0x7fe36809e2f0 msgr2=0x7fe3680681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 shutdown_connections 2024-04-26T21:47:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.496+0000 7fe367e00640 1 -- 172.21.15.106:0/622767212 wait complete. 2024-04-26T21:47:42.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:47:42.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.714+0000 7f483f200640 1 Processor -- start 2024-04-26T21:47:42.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.715+0000 7f483f200640 1 -- start start 2024-04-26T21:47:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.716+0000 7f483f200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48400ce690 0x7f484015df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.716+0000 7f483f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400d2ab0 0x7f48400d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.716+0000 7f483f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48400cd630 con 0x7f48400b24a0 2024-04-26T21:47:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.716+0000 7f483f200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f48400cd8d0 con 0x7f48400ce690 2024-04-26T21:47:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.716+0000 7f483f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f48400cdbb0 con 0x7f48400d2ab0 2024-04-26T21:47:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48400ce690 0x7f484015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48400ce690 0x7f484015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55164/0 (socket says 172.21.15.106:55164) 2024-04-26T21:47:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 -- 172.21.15.106:0/650197979 learned_addr learned my addr 172.21.15.106:0/650197979 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483e800640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400d2ab0 0x7f48400d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 -- 172.21.15.106:0/650197979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400d2ab0 msgr2=0x7f48400d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400d2ab0 0x7f48400d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 -- 172.21.15.106:0/650197979 >> v1:172.21.15.204:6789/0 conn(0x7f48400b24a0 legacy=0x7f48400d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.717+0000 7f483d400640 1 -- 172.21.15.106:0/650197979 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48400cde90 con 0x7f48400ce690 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.718+0000 7f483d400640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484015df20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4828009e50 tx=0x7f482802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ac8db05263b255 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.718+0000 7f483ca00640 1 -- 172.21.15.106:0/650197979 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828036070 con 0x7f48400ce690 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483ca00640 1 -- 172.21.15.106:0/650197979 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4828031600 con 0x7f48400ce690 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483ca00640 1 -- 172.21.15.106:0/650197979 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280319d0 con 0x7f48400ce690 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 -- 172.21.15.106:0/650197979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 msgr2=0x7f484015df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484015df20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4828009e50 tx=0x7f482802a5b0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 -- 172.21.15.106:0/650197979 shutdown_connections 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400d2ab0 0x7f48400d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 --2- 172.21.15.106:0/650197979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484015df20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 -- 172.21.15.106:0/650197979 >> 172.21.15.106:0/650197979 conn(0x7f484009ba50 msgr2=0x7f484009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 -- 172.21.15.106:0/650197979 shutdown_connections 2024-04-26T21:47:42.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.719+0000 7f483f200640 1 -- 172.21.15.106:0/650197979 wait complete. 2024-04-26T21:47:42.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.721+0000 7f483f200640 1 Processor -- start 2024-04-26T21:47:42.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.721+0000 7f483f200640 1 -- start start 2024-04-26T21:47:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 0x7f484014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48400d2ab0 0x7f484015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f484015a140 con 0x7f48400ce690 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4840159fc0 con 0x7f48400d2ab0 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f484015a2c0 con 0x7f48400b24a0 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.722+0000 7f483d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55174/0 (socket says 172.21.15.106:55174) 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 -- 172.21.15.106:0/3462616702 learned_addr learned my addr 172.21.15.106:0/3462616702 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483de00640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 0x7f484014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483e800640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48400d2ab0 0x7f484015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 -- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 msgr2=0x7f484014f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 0x7f484014f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 -- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48400d2ab0 msgr2=0x7f484015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48400d2ab0 0x7f484015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 -- 172.21.15.106:0/3462616702 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48400b0e80 con 0x7f48400ce690 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483de00640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 0x7f484014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.723+0000 7f483d400640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4828009e50 tx=0x7f4828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.724+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828036050 con 0x7f48400ce690 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.724+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4828042070 con 0x7f48400ce690 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.724+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48400b1160 con 0x7f48400ce690 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.724+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828007c70 con 0x7f48400ce690 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.724+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484006f1b0 con 0x7f48400ce690 2024-04-26T21:47:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.725+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4840069ad0 con 0x7f48400ce690 2024-04-26T21:47:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.726+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f4828047460 con 0x7f48400ce690 2024-04-26T21:47:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.727+0000 7f4836000640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 0x7f481c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.727+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f48280acfe0 con 0x7f48400ce690 2024-04-26T21:47:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.727+0000 7f483de00640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 0x7f481c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.728+0000 7f483de00640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 0x7f481c06ef90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4830003eb0 tx=0x7f4830009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.731+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4828045360 con 0x7f48400ce690 2024-04-26T21:47:42.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.946+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48400cebd0 con 0x7f48400ce690 2024-04-26T21:47:42.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.947+0000 7f4836000640 1 -- 172.21.15.106:0/3462616702 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f482807b0d0 con 0x7f48400ce690 2024-04-26T21:47:42.948 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:47:42.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 msgr2=0x7f481c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 0x7f481c06ef90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4830003eb0 tx=0x7f4830009040 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 msgr2=0x7f484006ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4828009e50 tx=0x7f4828004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 shutdown_connections 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48400d2ab0 0x7f484015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48400ce690 0x7f484006ec70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f481c06cad0 0x7f481c06ef90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 --2- 172.21.15.106:0/3462616702 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48400b24a0 0x7f484014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 >> 172.21.15.106:0/3462616702 conn(0x7f484009ba50 msgr2=0x7f48400ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.949+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 shutdown_connections 2024-04-26T21:47:42.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:42.950+0000 7f483f200640 1 -- 172.21.15.106:0/3462616702 wait complete. 2024-04-26T21:47:42.969 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-04-26T21:47:42.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:42.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T21:47:43.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[40500]: fuse finished with error 0 and tester_r 0 2024-04-26T21:47:43.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:47:43.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:43.057 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:47:43.103 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:43.104 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:43.104 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:43.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:43.160 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:43.160 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:47:43.160 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:43.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:43.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:43.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.434+0000 7f30c6e00640 1 Processor -- start 2024-04-26T21:47:43.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.435+0000 7f30c6e00640 1 -- start start 2024-04-26T21:47:43.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30c00d5990 0x7f30c0163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c6e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c00d3f50 con 0x7f30c015f1b0 2024-04-26T21:47:43.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c6e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c00d36b0 con 0x7f30c00b82b0 2024-04-26T21:47:43.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00d3990 con 0x7f30c00d5990 2024-04-26T21:47:43.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c5a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f30c015f1b0 0x7f30c0162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35682/0 (socket says 172.21.15.106:35682) 2024-04-26T21:47:43.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.436+0000 7f30c5a00640 1 -- 172.21.15.106:0/3405790849 learned_addr learned my addr 172.21.15.106:0/3405790849 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:43.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 812721618 0 0) 0x7f30c00d3f50 con 0x7f30c015f1b0 2024-04-26T21:47:43.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30a4003520 con 0x7f30c015f1b0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30c5000640 1 --2- 172.21.15.106:0/3405790849 >> v2:172.21.15.204:3300/0 conn(0x7f30c00d5990 0x7f30c0163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2884577578 0 0) 0x7f30a4003520 con 0x7f30c015f1b0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 >> v2:172.21.15.204:3300/0 conn(0x7f30c00d5990 msgr2=0x7f30c0163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 --2- 172.21.15.106:0/3405790849 >> v2:172.21.15.204:3300/0 conn(0x7f30c00d5990 0x7f30c0163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 >> v1:172.21.15.204:6789/0 conn(0x7f30c00b82b0 legacy=0x7f30c00b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00d3c70 con 0x7f30c015f1b0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f30b0001670 con 0x7f30c015f1b0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 >> v1:172.21.15.106:6789/0 conn(0x7f30c015f1b0 legacy=0x7f30c0162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30a40039b0 0x7f30a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.437+0000 7f30bfe00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 0x7f30a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d3990 con 0x7f30a40051c0 2024-04-26T21:47:43.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30a400c110 con 0x7f30a4004800 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f30a400c310 con 0x7f30a40039b0 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 0x7f30a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c5a00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30a40039b0 0x7f30a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c5000640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 -- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30a40039b0 msgr2=0x7f30a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30a40039b0 0x7f30a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 -- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a4004800 msgr2=0x7f30a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c6400640 1 -- 172.21.15.106:0/3405790849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a4003520 con 0x7f30a40051c0 2024-04-26T21:47:43.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.438+0000 7f30c5000640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:43.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.439+0000 7f30c6400640 1 -- 172.21.15.106:0/3405790849 reap_dead start 2024-04-26T21:47:43.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.439+0000 7f30c6400640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 0x7f30a400b9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f30b8001e50 tx=0x7f30a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109b461941c1a28f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8001a70 con 0x7f30a40051c0 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8003970 con 0x7f30a40051c0 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30bfe00640 1 -- 172.21.15.106:0/3405790849 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8003d60 con 0x7f30a40051c0 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30c6e00640 1 -- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 msgr2=0x7f30a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 0x7f30a400b9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f30b8001e50 tx=0x7f30a4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30c6e00640 1 -- 172.21.15.106:0/3405790849 shutdown_connections 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.440+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3405790849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30a40051c0 0x7f30a400b9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.441+0000 7f30c6e00640 1 -- 172.21.15.106:0/3405790849 >> 172.21.15.106:0/3405790849 conn(0x7f30c009e230 msgr2=0x7f30c00d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.441+0000 7f30c6e00640 1 -- 172.21.15.106:0/3405790849 shutdown_connections 2024-04-26T21:47:43.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.441+0000 7f30c6e00640 1 -- 172.21.15.106:0/3405790849 wait complete. 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.442+0000 7f30c6e00640 1 Processor -- start 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.443+0000 7f30c6e00640 1 -- start start 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.443+0000 7f30c6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 0x7f30c015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.443+0000 7f30c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30c015c5c0 0x7f30c01eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.443+0000 7f30c6e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30c0162b00 con 0x7f30c014f3e0 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d33a0 con 0x7f30a40051c0 2024-04-26T21:47:43.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f30c0162c80 con 0x7f30c015c5c0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55190/0 (socket says 172.21.15.106:55190) 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 -- 172.21.15.106:0/3646117151 learned_addr learned my addr 172.21.15.106:0/3646117151 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c5a00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 0x7f30c015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c5000640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30c015c5c0 0x7f30c01eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 -- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30c015c5c0 msgr2=0x7f30c01eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30c015c5c0 0x7f30c01eaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 -- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 msgr2=0x7f30c015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 0x7f30c015da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c6400640 1 -- 172.21.15.106:0/3646117151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c01ea460 con 0x7f30c014f3e0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.444+0000 7f30c5a00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 0x7f30c015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30c6400640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005550 tx=0x7f30b80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8001a70 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b802ac70 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8006d90 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c00c9f50 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.445+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c01ea9e0 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.446+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c01eabe0 con 0x7f30c014f3e0 2024-04-26T21:47:43.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.447+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f30b803b030 con 0x7f30c014f3e0 2024-04-26T21:47:43.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.448+0000 7f30bd600640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 0x7f30a006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.448+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f30b80aaec0 con 0x7f30c014f3e0 2024-04-26T21:47:43.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.448+0000 7f30c5a00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 0x7f30a006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.449+0000 7f30c5a00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 0x7f30a006ef60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30b0001200 tx=0x7f30b0001170 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.453+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30c01eabe0 con 0x7f30c014f3e0 2024-04-26T21:47:43.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.693+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c00693c0 con 0x7f30c014f3e0 2024-04-26T21:47:43.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.694+0000 7f30bd600640 1 -- 172.21.15.106:0/3646117151 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f30b8079070 con 0x7f30c014f3e0 2024-04-26T21:47:43.695 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:43.695 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:43.696 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-26T21:47:43.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 msgr2=0x7f30a006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 0x7f30a006ef60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30b0001200 tx=0x7f30b0001170 comp rx=0 tx=0).stop 2024-04-26T21:47:43.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 msgr2=0x7f30c00caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005550 tx=0x7f30b80014a0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 shutdown_connections 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30c014f3e0 0x7f30c00caa70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f30a006caa0 0x7f30a006ef60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30c015c5c0 0x7f30c01eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 --2- 172.21.15.106:0/3646117151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a40051c0 0x7f30c015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 >> 172.21.15.106:0/3646117151 conn(0x7f30c009e230 msgr2=0x7f30c00b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 shutdown_connections 2024-04-26T21:47:43.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.697+0000 7f30c6e00640 1 -- 172.21.15.106:0/3646117151 wait complete. 2024-04-26T21:47:43.716 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:43.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.916+0000 7f9d27200640 1 Processor -- start 2024-04-26T21:47:43.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.916+0000 7f9d27200640 1 -- start start 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.917+0000 7f9d27200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280d0e70 0x7f9d280d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d27200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d280b5660 0x7f9d280ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d27200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 0x7f9d280aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d27200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9d280d1790 con 0x7f9d280b5660 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d27200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9d280727f0 con 0x7f9d280b5d40 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d27200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d280b4a10 con 0x7f9d280d0e70 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d25400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d280b5660 0x7f9d280ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d25400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d280b5660 0x7f9d280ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55204/0 (socket says 172.21.15.106:55204) 2024-04-26T21:47:43.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d25400640 1 -- 172.21.15.106:0/4068855392 learned_addr learned my addr 172.21.15.106:0/4068855392 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d25e00640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280d0e70 0x7f9d280d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d25400640 1 -- 172.21.15.106:0/4068855392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280d0e70 msgr2=0x7f9d280d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.918+0000 7f9d26800640 1 --2- 172.21.15.106:0/4068855392 >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 0x7f9d280aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d25400640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280d0e70 0x7f9d280d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d25400640 1 -- 172.21.15.106:0/4068855392 >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 msgr2=0x7f9d280aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d25400640 1 --2- 172.21.15.106:0/4068855392 >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 0x7f9d280aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d25400640 1 -- 172.21.15.106:0/4068855392 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d280b4cf0 con 0x7f9d280b5660 2024-04-26T21:47:43.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.919+0000 7f9d26800640 1 --2- 172.21.15.106:0/4068855392 >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 0x7f9d280aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:43.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.920+0000 7f9d25400640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d280ae8c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d28072320 tx=0x7f9d1402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8232493b5e87b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.920+0000 7f9d24a00640 1 -- 172.21.15.106:0/4068855392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14037070 con 0x7f9d280b5660 2024-04-26T21:47:43.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.920+0000 7f9d24a00640 1 -- 172.21.15.106:0/4068855392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d14032440 con 0x7f9d280b5660 2024-04-26T21:47:43.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.920+0000 7f9d24a00640 1 -- 172.21.15.106:0/4068855392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14032810 con 0x7f9d280b5660 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 -- 172.21.15.106:0/4068855392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 msgr2=0x7f9d280ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d280ae8c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d28072320 tx=0x7f9d1402a140 comp rx=0 tx=0).stop 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 -- 172.21.15.106:0/4068855392 shutdown_connections 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 --2- 172.21.15.106:0/4068855392 >> v2:172.21.15.204:3300/0 conn(0x7f9d280b5d40 0x7f9d280aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d280ae8c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 --2- 172.21.15.106:0/4068855392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280d0e70 0x7f9d280d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 -- 172.21.15.106:0/4068855392 >> 172.21.15.106:0/4068855392 conn(0x7f9d2806c740 msgr2=0x7f9d2806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 -- 172.21.15.106:0/4068855392 shutdown_connections 2024-04-26T21:47:43.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.921+0000 7f9d27200640 1 -- 172.21.15.106:0/4068855392 wait complete. 2024-04-26T21:47:43.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.923+0000 7f9d27200640 1 Processor -- start 2024-04-26T21:47:43.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.923+0000 7f9d27200640 1 -- start start 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b5d40 0x7f9d28160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d280d0e70 0x7f9d2815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280b05b0 con 0x7f9d280b5660 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280b0430 con 0x7f9d280d0e70 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d27200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2815f1a0 con 0x7f9d280b5d40 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d25e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d25e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55210/0 (socket says 172.21.15.106:55210) 2024-04-26T21:47:43.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.924+0000 7f9d25e00640 1 -- 172.21.15.106:0/2465141515 learned_addr learned my addr 172.21.15.106:0/2465141515 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25400640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b5d40 0x7f9d28160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 -- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b5d40 msgr2=0x7f9d28160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d26800640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d280d0e70 0x7f9d2815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b5d40 0x7f9d28160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 -- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d280d0e70 msgr2=0x7f9d2815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d280d0e70 0x7f9d2815c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 -- 172.21.15.106:0/2465141515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d281e8950 con 0x7f9d280b5660 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d25e00640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d18007ef0 tx=0x7f9d1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18014620 con 0x7f9d280b5660 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d18019070 con 0x7f9d280b5660 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d281e8c30 con 0x7f9d280b5660 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18014a00 con 0x7f9d280b5660 2024-04-26T21:47:43.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.925+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d281e91f0 con 0x7f9d280b5660 2024-04-26T21:47:43.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.927+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2815ea00 con 0x7f9d280b5660 2024-04-26T21:47:43.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.927+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9d18014ba0 con 0x7f9d280b5660 2024-04-26T21:47:43.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.928+0000 7f9d1e000640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 0x7f9d0806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.929+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9d1808c870 con 0x7f9d280b5660 2024-04-26T21:47:43.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.929+0000 7f9d25400640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 0x7f9d0806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.929+0000 7f9d25400640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 0x7f9d0806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d14034000 tx=0x7f9d1402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:43.933+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d2815ea00 con 0x7f9d280b5660 2024-04-26T21:47:44.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.132+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d280693c0 con 0x7f9d280b5660 2024-04-26T21:47:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.134+0000 7f9d1e000640 1 -- 172.21.15.106:0/2465141515 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f9d1805aa10 con 0x7f9d280b5660 2024-04-26T21:47:44.136 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:44.136 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:44.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 msgr2=0x7f9d0806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 0x7f9d0806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d14034000 tx=0x7f9d1402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 msgr2=0x7f9d281e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d18007ef0 tx=0x7f9d1800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 shutdown_connections 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d280d0e70 0x7f9d2815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d0806caa0 0x7f9d0806ef60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b5d40 0x7f9d28160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 --2- 172.21.15.106:0/2465141515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d280b5660 0x7f9d281e99f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 >> 172.21.15.106:0/2465141515 conn(0x7f9d2806c740 msgr2=0x7f9d2815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 shutdown_connections 2024-04-26T21:47:44.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.137+0000 7f9d27200640 1 -- 172.21.15.106:0/2465141515 wait complete. 2024-04-26T21:47:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:12.735544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:13.898739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:44.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:44.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.341+0000 7f82c2a00640 1 Processor -- start 2024-04-26T21:47:44.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.342+0000 7f82c2a00640 1 -- start start 2024-04-26T21:47:44.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.343+0000 7f82c2a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82bc06ce80 0x7f82bc0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.343+0000 7f82c2a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82bc0af800 con 0x7f82bc15d100 2024-04-26T21:47:44.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.343+0000 7f82c2a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82bc0ae340 con 0x7f82bc0b5ca0 2024-04-26T21:47:44.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.343+0000 7f82c2a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f82bc0afaa0 con 0x7f82bc06ce80 2024-04-26T21:47:44.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bbe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82bc15d100 0x7f82bc06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35698/0 (socket says 172.21.15.106:35698) 2024-04-26T21:47:44.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bbe00640 1 -- 172.21.15.106:0/3331574621 learned_addr learned my addr 172.21.15.106:0/3331574621 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:44.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bb400640 1 --2- 172.21.15.106:0/3331574621 >> v2:172.21.15.106:3300/0 conn(0x7f82bc06ce80 0x7f82bc0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82baa00640 1 -- 172.21.15.106:0/3331574621 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726182440 0 0) 0x7f82bc0af800 con 0x7f82bc15d100 2024-04-26T21:47:44.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82baa00640 1 -- 172.21.15.106:0/3331574621 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f829c003520 con 0x7f82bc15d100 2024-04-26T21:47:44.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bb400640 1 -- 172.21.15.106:0/3331574621 >> v1:172.21.15.204:6789/0 conn(0x7f82bc0b5ca0 legacy=0x7f82bc0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bb400640 1 -- 172.21.15.106:0/3331574621 >> v1:172.21.15.106:6789/0 conn(0x7f82bc15d100 legacy=0x7f82bc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.344+0000 7f82bb400640 1 -- 172.21.15.106:0/3331574621 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc0afd80 con 0x7f82bc06ce80 2024-04-26T21:47:44.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.345+0000 7f82bb400640 1 --2- 172.21.15.106:0/3331574621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc0b3c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7beadf650511a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82baa00640 1 -- 172.21.15.106:0/3331574621 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4007740 con 0x7f82bc06ce80 2024-04-26T21:47:44.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82baa00640 1 -- 172.21.15.106:0/3331574621 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a40078e0 con 0x7f82bc06ce80 2024-04-26T21:47:44.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82baa00640 1 -- 172.21.15.106:0/3331574621 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4007cb0 con 0x7f82bc06ce80 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 -- 172.21.15.106:0/3331574621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 msgr2=0x7f82bc0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3331574621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc0b3c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a402a660 comp rx=0 tx=0).stop 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 -- 172.21.15.106:0/3331574621 shutdown_connections 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3331574621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc0b3c30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 -- 172.21.15.106:0/3331574621 >> 172.21.15.106:0/3331574621 conn(0x7f82bc0b2d50 msgr2=0x7f82bc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 -- 172.21.15.106:0/3331574621 shutdown_connections 2024-04-26T21:47:44.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.346+0000 7f82c2a00640 1 -- 172.21.15.106:0/3331574621 wait complete. 2024-04-26T21:47:44.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.348+0000 7f82c2a00640 1 Processor -- start 2024-04-26T21:47:44.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.348+0000 7f82c2a00640 1 -- start start 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82bc0b5ca0 0x7f82bc141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82bc15d100 0x7f82bc151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc09ee50 con 0x7f82bc06ce80 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc0b0340 con 0x7f82bc0b5ca0 2024-04-26T21:47:44.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82c2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc160b70 con 0x7f82bc15d100 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.349+0000 7f82bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55232/0 (socket says 172.21.15.106:55232) 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 -- 172.21.15.106:0/3400370455 learned_addr learned my addr 172.21.15.106:0/3400370455 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82c0a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82bc15d100 0x7f82bc151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bb400640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82bc0b5ca0 0x7f82bc141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 -- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82bc15d100 msgr2=0x7f82bc151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82bc15d100 0x7f82bc151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 -- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82bc0b5ca0 msgr2=0x7f82bc141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82bc0b5ca0 0x7f82bc141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 -- 172.21.15.106:0/3400370455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc14f8d0 con 0x7f82bc06ce80 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.350+0000 7f82bbe00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c410 tx=0x7f82ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.351+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac017070 con 0x7f82bc06ce80 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.351+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ac005ce0 con 0x7f82bc06ce80 2024-04-26T21:47:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.351+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0072f0 con 0x7f82bc06ce80 2024-04-26T21:47:44.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.351+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc14fcb0 con 0x7f82bc06ce80 2024-04-26T21:47:44.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.351+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc150270 con 0x7f82bc06ce80 2024-04-26T21:47:44.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.352+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc069650 con 0x7f82bc06ce80 2024-04-26T21:47:44.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.353+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f82ac01a070 con 0x7f82bc06ce80 2024-04-26T21:47:44.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.354+0000 7f82b3e00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 0x7f829806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.354+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f82ac08b430 con 0x7f82bc06ce80 2024-04-26T21:47:44.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.354+0000 7f82bb400640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 0x7f829806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.355+0000 7f82bb400640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 0x7f829806ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82a4034000 tx=0x7f82a402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.358+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82ac059570 con 0x7f82bc06ce80 2024-04-26T21:47:44.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.554+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82bc0d1890 con 0x7f82bc06ce80 2024-04-26T21:47:44.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.554+0000 7f82b3e00640 1 -- 172.21.15.106:0/3400370455 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f82ac058e70 con 0x7f82bc06ce80 2024-04-26T21:47:44.555 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 msgr2=0x7f829806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 0x7f829806ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82a4034000 tx=0x7f82a402f000 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 msgr2=0x7f82bc14f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c410 tx=0x7f82ac00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 shutdown_connections 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f829806caa0 0x7f829806ef60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82bc15d100 0x7f82bc151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82bc0b5ca0 0x7f82bc141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 --2- 172.21.15.106:0/3400370455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82bc06ce80 0x7f82bc14f390 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.557+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 >> 172.21.15.106:0/3400370455 conn(0x7f82bc0b2d50 msgr2=0x7f82bc0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.558+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 shutdown_connections 2024-04-26T21:47:44.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.558+0000 7f82c2a00640 1 -- 172.21.15.106:0/3400370455 wait complete. 2024-04-26T21:47:44.577 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:44.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.771+0000 7f4bc3200640 1 Processor -- start 2024-04-26T21:47:44.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.771+0000 7f4bc3200640 1 -- start start 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc3200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4bc415ee60 0x7f4bc40b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc3200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc415d970 con 0x7f4bc415cf40 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc3200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc40b4fe0 con 0x7f4bc415c450 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc3200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc40b52c0 con 0x7f4bc415ee60 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc2800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4bc415cf40 0x7f4bc4165a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35710/0 (socket says 172.21.15.106:35710) 2024-04-26T21:47:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.772+0000 7f4bc2800640 1 -- 172.21.15.106:0/3939908854 learned_addr learned my addr 172.21.15.106:0/3939908854 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956796618 0 0) 0x7f4bc415d970 con 0x7f4bc415cf40 2024-04-26T21:47:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ba8005730 con 0x7f4bc415cf40 2024-04-26T21:47:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/3939908854 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415ee60 0x7f4bc40b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1917091646 0 0) 0x7f4ba8005730 con 0x7f4bc415cf40 2024-04-26T21:47:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415ee60 msgr2=0x7f4bc40b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 --2- 172.21.15.106:0/3939908854 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415ee60 0x7f4bc40b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 >> v1:172.21.15.204:6789/0 conn(0x7f4bc415c450 legacy=0x7f4bc415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc40b55a0 con 0x7f4bc415cf40 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f4bbc0019f0 con 0x7f4bc415cf40 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 >> v1:172.21.15.106:6789/0 conn(0x7f4bc415cf40 legacy=0x7f4bc4165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ba8005c30 0x7f4ba8006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 0x7f4ba8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba80074f0 0x7f4ba800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc40b52c0 con 0x7f4ba8006a10 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba800e440 con 0x7f4ba80074f0 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.773+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba800e640 con 0x7f4ba8005c30 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 0x7f4ba8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bc2800640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba80074f0 0x7f4ba800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bc1e00640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ba8005c30 0x7f4ba8006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 -- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ba8005c30 msgr2=0x7f4ba8006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ba8005c30 0x7f4ba8006050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 -- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba80074f0 msgr2=0x7f4ba800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba80074f0 0x7f4ba800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bbbe00640 1 -- 172.21.15.106:0/3939908854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba8005730 con 0x7f4ba8006a10 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bc2800640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba80074f0 0x7f4ba800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.774+0000 7f4bc2800640 1 -- 172.21.15.106:0/3939908854 reap_dead start 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.775+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 0x7f4ba8006e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4bc415f2f0 tx=0x7f4ba8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0134e276695e1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.775+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac030070 con 0x7f4ba8006a10 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.775+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac02ad80 con 0x7f4ba8006a10 2024-04-26T21:47:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.775+0000 7f4bc1400640 1 -- 172.21.15.106:0/3939908854 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00b610 con 0x7f4ba8006a10 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 -- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 msgr2=0x7f4ba8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 0x7f4ba8006e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4bc415f2f0 tx=0x7f4ba8007a40 comp rx=0 tx=0).stop 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 -- 172.21.15.106:0/3939908854 shutdown_connections 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 --2- 172.21.15.106:0/3939908854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ba8006a10 0x7f4ba8006e90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 -- 172.21.15.106:0/3939908854 >> 172.21.15.106:0/3939908854 conn(0x7f4bc40b2900 msgr2=0x7f4bc40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 -- 172.21.15.106:0/3939908854 shutdown_connections 2024-04-26T21:47:44.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.776+0000 7f4bc3200640 1 -- 172.21.15.106:0/3939908854 wait complete. 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.778+0000 7f4bc3200640 1 Processor -- start 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.778+0000 7f4bc3200640 1 -- start start 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 0x7f4bc4152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc41f15b0 0x7f4bc41f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc40b4aa0 con 0x7f4bc41588d0 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc40b4920 con 0x7f4ba8006a10 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc40b4c20 con 0x7f4bc41f15b0 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55254/0 (socket says 172.21.15.106:55254) 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 -- 172.21.15.106:0/2101248898 learned_addr learned my addr 172.21.15.106:0/2101248898 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc2800640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc41f15b0 0x7f4bc41f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 -- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc41f15b0 msgr2=0x7f4bc41f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc41f15b0 0x7f4bc41f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 -- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 msgr2=0x7f4bc4152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc1e00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 0x7f4bc4152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 0x7f4bc4152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bbbe00640 1 -- 172.21.15.106:0/2101248898 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc41f1fa0 con 0x7f4bc41588d0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.779+0000 7f4bc1e00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 0x7f4bc4152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bbbe00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00bb00 tx=0x7f4bac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac043020 con 0x7f4bc41588d0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac03f680 con 0x7f4bc41588d0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00b610 con 0x7f4bc41588d0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc41534b0 con 0x7f4bc41588d0 2024-04-26T21:47:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.780+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc4153a70 con 0x7f4bc41588d0 2024-04-26T21:47:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.781+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc40693c0 con 0x7f4bc41588d0 2024-04-26T21:47:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.782+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f4bac046020 con 0x7f4bc41588d0 2024-04-26T21:47:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.783+0000 7f4bba000640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 0x7f4ba006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.783+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4bac0aca60 con 0x7f4bc41588d0 2024-04-26T21:47:44.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.783+0000 7f4bc1e00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 0x7f4ba006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.784+0000 7f4bc1e00640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 0x7f4ba006ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40cfe80 tx=0x7f4bb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.788+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bac04d030 con 0x7f4bc41588d0 2024-04-26T21:47:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.986+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bc40d0560 con 0x7f4bc41588d0 2024-04-26T21:47:44.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.987+0000 7f4bba000640 1 -- 172.21.15.106:0/2101248898 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f4bac07ac00 con 0x7f4bc41588d0 2024-04-26T21:47:44.988 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 msgr2=0x7f4ba006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 0x7f4ba006ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40cfe80 tx=0x7f4bb4009040 comp rx=0 tx=0).stop 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 msgr2=0x7f4bc4158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00bb00 tx=0x7f4bac004730 comp rx=0 tx=0).stop 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 shutdown_connections 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f4ba006cad0 0x7f4ba006ef90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc41f15b0 0x7f4bc41f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc41588d0 0x7f4bc4158d50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 --2- 172.21.15.106:0/2101248898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8006a10 0x7f4bc4152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 >> 172.21.15.106:0/2101248898 conn(0x7f4bc40b2900 msgr2=0x7f4bc40b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 shutdown_connections 2024-04-26T21:47:44.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:44.990+0000 7f4bc3200640 1 -- 172.21.15.106:0/2101248898 wait complete. 2024-04-26T21:47:45.010 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:45.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.211+0000 7fc8cea00640 1 Processor -- start 2024-04-26T21:47:45.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.211+0000 7fc8cea00640 1 -- start start 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8cea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8c80b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8cea00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80af800 con 0x7fc8c815d100 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8cea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80ae340 con 0x7fc8c80b5ca0 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8cea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c80afaa0 con 0x7fc8c806ce80 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8c7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc8c815d100 0x7fc8c806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35722/0 (socket says 172.21.15.106:35722) 2024-04-26T21:47:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.213+0000 7fc8c7e00640 1 -- 172.21.15.106:0/4153615862 learned_addr learned my addr 172.21.15.106:0/4153615862 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c7400640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8c80b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726825137 0 0) 0x7fc8c80af800 con 0x7fc8c815d100 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8ac003520 con 0x7fc8c815d100 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 180907090 0 0) 0x7fc8ac003520 con 0x7fc8c815d100 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 msgr2=0x7fc8c80b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8c80b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 >> v1:172.21.15.204:6789/0 conn(0x7fc8c80b5ca0 legacy=0x7fc8c80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80afd80 con 0x7fc8c815d100 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fc8b8001670 con 0x7fc8c815d100 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 >> v1:172.21.15.106:6789/0 conn(0x7fc8c815d100 legacy=0x7fc8c806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8ac0039e0 0x7fc8ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8ac004800 0x7fc8ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80afaa0 con 0x7fc8ac0039e0 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00ba70 con 0x7fc8ac004800 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.214+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac00bc70 con 0x7fc8c806ce80 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8ac0039e0 0x7fc8ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7400640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8ac004800 0x7fc8ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8cca00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 -- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 msgr2=0x7fc8ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c806ce80 0x7fc8ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 -- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8ac004800 msgr2=0x7fc8ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8ac004800 0x7fc8ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.215+0000 7fc8c7e00640 1 -- 172.21.15.106:0/4153615862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac003520 con 0x7fc8ac0039e0 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.217+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8ac0039e0 0x7fc8ac003e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c210 tx=0x7fc8ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234680f9c6619c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.217+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007d10 con 0x7fc8ac0039e0 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.217+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8005ad0 con 0x7fc8ac0039e0 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.217+0000 7fc8c6a00640 1 -- 172.21.15.106:0/4153615862 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007070 con 0x7fc8ac0039e0 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 -- 172.21.15.106:0/4153615862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8ac0039e0 msgr2=0x7fc8ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 --2- 172.21.15.106:0/4153615862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8ac0039e0 0x7fc8ac003e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c210 tx=0x7fc8ac005830 comp rx=0 tx=0).stop 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cca00640 1 -- 172.21.15.106:0/4153615862 reap_dead start 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 -- 172.21.15.106:0/4153615862 shutdown_connections 2024-04-26T21:47:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 -- 172.21.15.106:0/4153615862 >> 172.21.15.106:0/4153615862 conn(0x7fc8c80b2d50 msgr2=0x7fc8c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 -- 172.21.15.106:0/4153615862 shutdown_connections 2024-04-26T21:47:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.218+0000 7fc8cea00640 1 -- 172.21.15.106:0/4153615862 wait complete. 2024-04-26T21:47:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.220+0000 7fc8cea00640 1 Processor -- start 2024-04-26T21:47:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.220+0000 7fc8cea00640 1 -- start start 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c8153060 0x7fc8c80b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c8153440 0x7fc8c80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c809ee50 con 0x7fc8c8158d50 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80b0340 con 0x7fc8c8153440 2024-04-26T21:47:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c8160b70 con 0x7fc8c8153060 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55274/0 (socket says 172.21.15.106:55274) 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 -- 172.21.15.106:0/3465280135 learned_addr learned my addr 172.21.15.106:0/3465280135 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c8153060 0x7fc8c80b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8c7400640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c8153440 0x7fc8c80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 -- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c8153060 msgr2=0x7fc8c80b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c8153060 0x7fc8c80b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.221+0000 7fc8cca00640 1 -- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c8153440 msgr2=0x7fc8c80b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8cca00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c8153440 0x7fc8c80b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8cca00640 1 -- 172.21.15.106:0/3465280135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c8159a80 con 0x7fc8c8158d50 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8cca00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80d1d90 tx=0x7fc8c000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c001a070 con 0x7fc8c8158d50 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0006d60 con 0x7fc8c8158d50 2024-04-26T21:47:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c8159d60 con 0x7fc8c8158d50 2024-04-26T21:47:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0003b90 con 0x7fc8c8158d50 2024-04-26T21:47:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.222+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c815a320 con 0x7fc8c8158d50 2024-04-26T21:47:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.223+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8069650 con 0x7fc8c8158d50 2024-04-26T21:47:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.224+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc8c001d070 con 0x7fc8c8158d50 2024-04-26T21:47:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.225+0000 7fc8bfe00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 0x7fc8a806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.225+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc8c008b4d0 con 0x7fc8c8158d50 2024-04-26T21:47:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.226+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 0x7fc8a806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.226+0000 7fc8c7e00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 0x7fc8a806ef60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8032000 tx=0x7fc8b802d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.230+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8c00595e0 con 0x7fc8c8158d50 2024-04-26T21:47:45.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.423+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc8c809e760 con 0x7fc8c8158d50 2024-04-26T21:47:45.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.423+0000 7fc8bfe00640 1 -- 172.21.15.106:0/3465280135 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fc8c0058ee0 con 0x7fc8c8158d50 2024-04-26T21:47:45.424 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 msgr2=0x7fc8a806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 0x7fc8a806ef60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8032000 tx=0x7fc8b802d000 comp rx=0 tx=0).stop 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 msgr2=0x7fc8c8159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80d1d90 tx=0x7fc8c000fd40 comp rx=0 tx=0).stop 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 shutdown_connections 2024-04-26T21:47:45.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8c8158d50 0x7fc8c8159200 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c8153440 0x7fc8c80b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc8a806caa0 0x7fc8a806ef60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 --2- 172.21.15.106:0/3465280135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c8153060 0x7fc8c80b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 >> 172.21.15.106:0/3465280135 conn(0x7fc8c80b2d50 msgr2=0x7fc8c80b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 shutdown_connections 2024-04-26T21:47:45.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.426+0000 7fc8cea00640 1 -- 172.21.15.106:0/3465280135 wait complete. 2024-04-26T21:47:45.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:45.506 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:45.513 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.625+0000 7f5bf0a00640 1 Processor -- start 2024-04-26T21:47:45.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.626+0000 7f5bf0a00640 1 -- start start 2024-04-26T21:47:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.627+0000 7f5bf0a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5bec15f440 0x7f5bec0d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.627+0000 7f5bf0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec09c160 0x7f5bec0d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.627+0000 7f5bf0a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bec163c60 con 0x7f5bec1637a0 2024-04-26T21:47:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.627+0000 7f5bf0a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5bec0d1b80 con 0x7f5bec15f440 2024-04-26T21:47:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.627+0000 7f5bf0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec0d1250 con 0x7f5bec09c160 2024-04-26T21:47:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5bec15f440 0x7f5bec0d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5bec15f440 0x7f5bec0d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55284/0 (socket says 172.21.15.106:55284) 2024-04-26T21:47:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 -- 172.21.15.106:0/3597533174 learned_addr learned my addr 172.21.15.106:0/3597533174 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5bea800640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec09c160 0x7f5bec0d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 -- 172.21.15.106:0/3597533174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec09c160 msgr2=0x7f5bec0d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec09c160 0x7f5bec0d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 -- 172.21.15.106:0/3597533174 >> v1:172.21.15.204:6789/0 conn(0x7f5bec1637a0 legacy=0x7f5bec09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.628+0000 7f5be9e00640 1 -- 172.21.15.106:0/3597533174 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec0d1530 con 0x7f5bec15f440 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.629+0000 7f5be9e00640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec15f440 0x7f5bec0d3990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009870 tx=0x7f5bdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d839f9602cdeb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.629+0000 7f5be9400640 1 -- 172.21.15.106:0/3597533174 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc037070 con 0x7f5bec15f440 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.629+0000 7f5be9400640 1 -- 172.21.15.106:0/3597533174 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bdc032440 con 0x7f5bec15f440 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.629+0000 7f5be9400640 1 -- 172.21.15.106:0/3597533174 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc032810 con 0x7f5bec15f440 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 -- 172.21.15.106:0/3597533174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec15f440 msgr2=0x7f5bec0d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec15f440 0x7f5bec0d3990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009870 tx=0x7f5bdc02a270 comp rx=0 tx=0).stop 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 -- 172.21.15.106:0/3597533174 shutdown_connections 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec09c160 0x7f5bec0d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/3597533174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec15f440 0x7f5bec0d3990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 -- 172.21.15.106:0/3597533174 >> 172.21.15.106:0/3597533174 conn(0x7f5bec0d2e70 msgr2=0x7f5bec0d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 -- 172.21.15.106:0/3597533174 shutdown_connections 2024-04-26T21:47:45.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.630+0000 7f5bf0a00640 1 -- 172.21.15.106:0/3597533174 wait complete. 2024-04-26T21:47:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.632+0000 7f5bf0a00640 1 Processor -- start 2024-04-26T21:47:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.632+0000 7f5bf0a00640 1 -- start start 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 0x7f5bec09f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec1637a0 0x7f5bec09fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0702b0 con 0x7f5bec09c160 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec06d680 con 0x7f5bec15f440 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5bf0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec070430 con 0x7f5bec1637a0 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55288/0 (socket says 172.21.15.106:55288) 2024-04-26T21:47:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 -- 172.21.15.106:0/1151867041 learned_addr learned my addr 172.21.15.106:0/1151867041 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 -- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec1637a0 msgr2=0x7f5bec09fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec1637a0 0x7f5bec09fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 -- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 msgr2=0x7f5bec09f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be1400640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 0x7f5bec09f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 0x7f5bec09f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be9e00640 1 -- 172.21.15.106:0/1151867041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec14e2a0 con 0x7f5bec09c160 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.634+0000 7f5be1400640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 0x7f5bec09f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5be9e00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc02a2b0 tx=0x7f5bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc041020 con 0x7f5bec09c160 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0306c0 con 0x7f5bec09c160 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc037040 con 0x7f5bec09c160 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec0c8590 con 0x7f5bec09c160 2024-04-26T21:47:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.635+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec14e540 con 0x7f5bec09c160 2024-04-26T21:47:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.636+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec069540 con 0x7f5bec09c160 2024-04-26T21:47:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.637+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5bdc044020 con 0x7f5bec09c160 2024-04-26T21:47:45.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.638+0000 7f5be2a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 0x7f5bc806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.638+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5bdc0ac630 con 0x7f5bec09c160 2024-04-26T21:47:45.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.638+0000 7f5be1400640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 0x7f5bc806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.639+0000 7f5be1400640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 0x7f5bc806ef60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5bec15e1f0 tx=0x7f5bd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.643+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5bdc04b030 con 0x7f5bec09c160 2024-04-26T21:47:45.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.866+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bec0b8df0 con 0x7f5bec09c160 2024-04-26T21:47:45.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.867+0000 7f5be2a00640 1 -- 172.21.15.106:0/1151867041 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f5bdc07a870 con 0x7f5bec09c160 2024-04-26T21:47:45.868 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:45.868 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:45.868 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 msgr2=0x7f5bc806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 0x7f5bc806ef60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5bec15e1f0 tx=0x7f5bd4009040 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 msgr2=0x7f5bec0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc02a2b0 tx=0x7f5bdc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 shutdown_connections 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5bc806caa0 0x7f5bc806ef60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bec1637a0 0x7f5bec09fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bec15f440 0x7f5bec09f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 --2- 172.21.15.106:0/1151867041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bec09c160 0x7f5bec0b3a20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.869+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 >> 172.21.15.106:0/1151867041 conn(0x7f5bec0d2e70 msgr2=0x7f5bec0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.870+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 shutdown_connections 2024-04-26T21:47:45.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:45.870+0000 7f5bf0a00640 1 -- 172.21.15.106:0/1151867041 wait complete. 2024-04-26T21:47:45.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:46.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.124+0000 7f4a06a00640 1 Processor -- start 2024-04-26T21:47:46.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.125+0000 7f4a06a00640 1 -- start start 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a06a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4a0009ba50 0x7f4a0009be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a06a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a000b58a0 0x7f4a0006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a06a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0009c370 con 0x7f4a0006f230 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a06a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4a00070420 con 0x7f4a000b58a0 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a06a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0006e790 con 0x7f4a0009ba50 2024-04-26T21:47:46.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f49f7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a000b58a0 0x7f4a0006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a04a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4a0006f230 0x7f4a0006d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35732/0 (socket says 172.21.15.106:35732) 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.126+0000 7f4a04a00640 1 -- 172.21.15.106:0/2825431616 learned_addr learned my addr 172.21.15.106:0/2825431616 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49ffe00640 1 --2- 172.21.15.106:0/2825431616 >> v2:172.21.15.204:3300/0 conn(0x7f4a0009ba50 0x7f4a0009be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49f7400640 1 -- 172.21.15.106:0/2825431616 >> v2:172.21.15.204:3300/0 conn(0x7f4a0009ba50 msgr2=0x7f4a0009be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49f7400640 1 --2- 172.21.15.106:0/2825431616 >> v2:172.21.15.204:3300/0 conn(0x7f4a0009ba50 0x7f4a0009be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49f7400640 1 -- 172.21.15.106:0/2825431616 >> v1:172.21.15.106:6789/0 conn(0x7f4a0006f230 legacy=0x7f4a0006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49f7400640 1 -- 172.21.15.106:0/2825431616 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a00070670 con 0x7f4a000b58a0 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.127+0000 7f49ff400640 1 -- 172.21.15.106:0/2825431616 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234369532 0 0) 0x7f4a0009c370 con 0x7f4a0006f230 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.128+0000 7f49f7400640 1 --2- 172.21.15.106:0/2825431616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a000b58a0 0x7f4a0006d460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a000767a0 tx=0x7f49e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca167d9eeccc0cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.128+0000 7f49ff400640 1 -- 172.21.15.106:0/2825431616 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8037070 con 0x7f4a000b58a0 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.128+0000 7f49ff400640 1 -- 172.21.15.106:0/2825431616 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8032440 con 0x7f4a000b58a0 2024-04-26T21:47:46.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.128+0000 7f49ff400640 1 -- 172.21.15.106:0/2825431616 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e802ae60 con 0x7f4a000b58a0 2024-04-26T21:47:46.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 -- 172.21.15.106:0/2825431616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a000b58a0 msgr2=0x7f4a0006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 --2- 172.21.15.106:0/2825431616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a000b58a0 0x7f4a0006d460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a000767a0 tx=0x7f49e802a110 comp rx=0 tx=0).stop 2024-04-26T21:47:46.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 -- 172.21.15.106:0/2825431616 shutdown_connections 2024-04-26T21:47:46.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 --2- 172.21.15.106:0/2825431616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a000b58a0 0x7f4a0006d460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 --2- 172.21.15.106:0/2825431616 >> v2:172.21.15.204:3300/0 conn(0x7f4a0009ba50 0x7f4a0009be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 -- 172.21.15.106:0/2825431616 >> 172.21.15.106:0/2825431616 conn(0x7f4a000b4b70 msgr2=0x7f4a000b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 -- 172.21.15.106:0/2825431616 shutdown_connections 2024-04-26T21:47:46.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.129+0000 7f4a06a00640 1 -- 172.21.15.106:0/2825431616 wait complete. 2024-04-26T21:47:46.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.131+0000 7f4a06a00640 1 Processor -- start 2024-04-26T21:47:46.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.131+0000 7f4a06a00640 1 -- start start 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a0006f230 0x7f4a000ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 0x7f4a000c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0006ecc0 con 0x7f4a0009ba50 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0006e9e0 con 0x7f4a000b58a0 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f4a06a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a000cf440 con 0x7f4a0006f230 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f49f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f49f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55318/0 (socket says 172.21.15.106:55318) 2024-04-26T21:47:46.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.132+0000 7f49f7400640 1 -- 172.21.15.106:0/506923950 learned_addr learned my addr 172.21.15.106:0/506923950 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49ffe00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a0006f230 0x7f4a000ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f4a04a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 0x7f4a000c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 -- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a0006f230 msgr2=0x7f4a000ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a0006f230 0x7f4a000ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 -- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 msgr2=0x7f4a000c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 0x7f4a000c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 -- 172.21.15.106:0/506923950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a000cb740 con 0x7f4a0009ba50 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f4a04a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 0x7f4a000c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:46.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49f7400640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49e8035040 tx=0x7f49e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8037050 con 0x7f4a0009ba50 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.133+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8038040 con 0x7f4a0009ba50 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.134+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8006e10 con 0x7f4a0009ba50 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.134+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a000cba20 con 0x7f4a0009ba50 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.134+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a000cc500 con 0x7f4a0009ba50 2024-04-26T21:47:46.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.135+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a00069540 con 0x7f4a0009ba50 2024-04-26T21:47:46.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.136+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f49e8030980 con 0x7f4a0009ba50 2024-04-26T21:47:46.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.137+0000 7f49fcc00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 0x7f49dc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.137+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f49e80aca70 con 0x7f4a0009ba50 2024-04-26T21:47:46.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.137+0000 7f49ffe00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 0x7f49dc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.138+0000 7f49ffe00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 0x7f49dc06ef60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e90 tx=0x7f49f000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.141+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49e807ad30 con 0x7f4a0009ba50 2024-04-26T21:47:46.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.329+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a000ca9e0 con 0x7f4a0009ba50 2024-04-26T21:47:46.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.331+0000 7f49fcc00640 1 -- 172.21.15.106:0/506923950 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f49e807a630 con 0x7f4a0009ba50 2024-04-26T21:47:46.332 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:46.333 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 msgr2=0x7f49dc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 0x7f49dc06ef60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e90 tx=0x7f49f000b040 comp rx=0 tx=0).stop 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 msgr2=0x7f4a000c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49e8035040 tx=0x7f49e80046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 shutdown_connections 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a000b58a0 0x7f4a000c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009ba50 0x7f4a000c6ff0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f49dc06caa0 0x7f49dc06ef60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 --2- 172.21.15.106:0/506923950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a0006f230 0x7f4a000ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 >> 172.21.15.106:0/506923950 conn(0x7f4a000b4b70 msgr2=0x7f4a000cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 shutdown_connections 2024-04-26T21:47:46.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.334+0000 7f4a06a00640 1 -- 172.21.15.106:0/506923950 wait complete. 2024-04-26T21:47:46.354 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:47:46.354 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:46.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.542+0000 7f6bbbc00640 1 Processor -- start 2024-04-26T21:47:46.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.542+0000 7f6bbbc00640 1 -- start start 2024-04-26T21:47:46.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.543+0000 7f6bbbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406e130 0x7f6bb40ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.543+0000 7f6bbbc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bb40737c0 0x7f6bb406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bbbc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb40ae370 con 0x7f6bb406fb50 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bbbc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb40ae5f0 con 0x7f6bb40737c0 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bbbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb4072010 con 0x7f6bb406e130 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bb40737c0 0x7f6bb406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bb40737c0 0x7f6bb406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55324/0 (socket says 172.21.15.106:55324) 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 -- 172.21.15.106:0/2275380852 learned_addr learned my addr 172.21.15.106:0/2275380852 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9200640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406e130 0x7f6bb40ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 -- 172.21.15.106:0/2275380852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406e130 msgr2=0x7f6bb40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406e130 0x7f6bb40ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 -- 172.21.15.106:0/2275380852 >> v1:172.21.15.204:6789/0 conn(0x7f6bb406fb50 legacy=0x7f6bb40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.544+0000 7f6bb9c00640 1 -- 172.21.15.106:0/2275380852 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb40722f0 con 0x7f6bb40737c0 2024-04-26T21:47:46.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.545+0000 7f6bb9c00640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40737c0 0x7f6bb406e510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009e50 tx=0x7f6bac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3168adf559479f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bb3400640 1 -- 172.21.15.106:0/2275380852 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac036070 con 0x7f6bb40737c0 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bb3400640 1 -- 172.21.15.106:0/2275380852 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac031600 con 0x7f6bb40737c0 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bb3400640 1 -- 172.21.15.106:0/2275380852 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0319d0 con 0x7f6bb40737c0 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 -- 172.21.15.106:0/2275380852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40737c0 msgr2=0x7f6bb406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40737c0 0x7f6bb406e510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009e50 tx=0x7f6bac02a6c0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 -- 172.21.15.106:0/2275380852 shutdown_connections 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40737c0 0x7f6bb406e510 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/2275380852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406e130 0x7f6bb40ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 -- 172.21.15.106:0/2275380852 >> 172.21.15.106:0/2275380852 conn(0x7f6bb40b2d50 msgr2=0x7f6bb40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.546+0000 7f6bbbc00640 1 -- 172.21.15.106:0/2275380852 shutdown_connections 2024-04-26T21:47:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.547+0000 7f6bbbc00640 1 -- 172.21.15.106:0/2275380852 wait complete. 2024-04-26T21:47:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.548+0000 7f6bbbc00640 1 Processor -- start 2024-04-26T21:47:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.548+0000 7f6bbbc00640 1 -- start start 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406fb50 0x7f6bb4143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 0x7f6bb40cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4072b90 con 0x7f6bb406e130 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40728b0 con 0x7f6bb40737c0 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bbbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb40b4810 con 0x7f6bb406fb50 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bb9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bb9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55332/0 (socket says 172.21.15.106:55332) 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.549+0000 7f6bb9200640 1 -- 172.21.15.106:0/3929983607 learned_addr learned my addr 172.21.15.106:0/3929983607 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:46.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb3e00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406fb50 0x7f6bb4143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9c00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 0x7f6bb40cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 -- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406fb50 msgr2=0x7f6bb4143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406fb50 0x7f6bb4143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 -- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 msgr2=0x7f6bb40cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 0x7f6bb40cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 -- 172.21.15.106:0/3929983607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb40cfed0 con 0x7f6bb406e130 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9c00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 0x7f6bb40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb9200640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d620 tx=0x7f6ba800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.550+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800b620 con 0x7f6bb406e130 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.551+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba8019070 con 0x7f6bb406e130 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.551+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ba00 con 0x7f6bb406e130 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.551+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb4144430 con 0x7f6bb406e130 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.551+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41477d0 con 0x7f6bb406e130 2024-04-26T21:47:46.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.552+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb4069540 con 0x7f6bb406e130 2024-04-26T21:47:46.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.553+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6ba800bba0 con 0x7f6bb406e130 2024-04-26T21:47:46.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.554+0000 7f6bb0c00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.554+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6ba808c300 con 0x7f6bb406e130 2024-04-26T21:47:46.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.554+0000 7f6bb3e00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.555+0000 7f6bb3e00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40cd500 tx=0x7f6ba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.558+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6bb40cfed0 con 0x7f6bb406e130 2024-04-26T21:47:46.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.801+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bb415cc40 con 0x7f6bb406e130 2024-04-26T21:47:46.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.802+0000 7f6bb0c00640 1 -- 172.21.15.106:0/3929983607 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f6ba805a410 con 0x7f6bb406e130 2024-04-26T21:47:46.803 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:46.803 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:46.803 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 msgr2=0x7f6b9406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40cd500 tx=0x7f6ba0009040 comp rx=0 tx=0).stop 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 msgr2=0x7f6bb40cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d620 tx=0x7f6ba800dae0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 shutdown_connections 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40737c0 0x7f6bb40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb406fb50 0x7f6bb4143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 --2- 172.21.15.106:0/3929983607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb406e130 0x7f6bb40cf990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 >> 172.21.15.106:0/3929983607 conn(0x7f6bb40b2d50 msgr2=0x7f6bb40aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 shutdown_connections 2024-04-26T21:47:46.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:46.805+0000 7f6bbbc00640 1 -- 172.21.15.106:0/3929983607 wait complete. 2024-04-26T21:47:46.823 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:47.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.019+0000 7f0b1f800640 1 Processor -- start 2024-04-26T21:47:47.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.020+0000 7f0b1f800640 1 -- start start 2024-04-26T21:47:47.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.021+0000 7f0b1f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b1806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.021+0000 7f0b1f800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0b1815cfd0 0x7f0b18073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.021+0000 7f0b1f800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1806d090 con 0x7f0b1815cb70 2024-04-26T21:47:47.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.021+0000 7f0b1f800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0b180af460 con 0x7f0b1815cfd0 2024-04-26T21:47:47.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.021+0000 7f0b1f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180ae850 con 0x7f0b1806c770 2024-04-26T21:47:47.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b17e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b1815cb70 0x7f0b18073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35742/0 (socket says 172.21.15.106:35742) 2024-04-26T21:47:47.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b17e00640 1 -- 172.21.15.106:0/1074849383 learned_addr learned my addr 172.21.15.106:0/1074849383 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b1d800640 1 --2- 172.21.15.106:0/1074849383 >> v2:172.21.15.204:3300/0 conn(0x7f0b1815cfd0 0x7f0b18073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070809450 0 0) 0x7f0b1806d090 con 0x7f0b1815cb70 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b00003520 con 0x7f0b1815cb70 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b1ce00640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b1806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.022+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2428625447 0 0) 0x7f0b00003520 con 0x7f0b1815cb70 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 msgr2=0x7f0b1806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b1806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 >> v2:172.21.15.204:3300/0 conn(0x7f0b1815cfd0 msgr2=0x7f0b18073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 --2- 172.21.15.106:0/1074849383 >> v2:172.21.15.204:3300/0 conn(0x7f0b1815cfd0 0x7f0b18073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b180aeb30 con 0x7f0b1815cb70 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f0b04001670 con 0x7f0b1815cb70 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 >> v1:172.21.15.106:6789/0 conn(0x7f0b1815cb70 legacy=0x7f0b18073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180af460 con 0x7f0b00004d40 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180ae850 con 0x7f0b00003f30 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0000fe20 con 0x7f0b1806c770 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b1d800640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b1ce00640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.023+0000 7f0b17e00640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.024+0000 7f0b1d800640 1 -- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 msgr2=0x7f0b000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.024+0000 7f0b1d800640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b1806c770 0x7f0b000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.024+0000 7f0b1d800640 1 -- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b00003f30 msgr2=0x7f0b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.024+0000 7f0b1d800640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.024+0000 7f0b1d800640 1 -- 172.21.15.106:0/1074849383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b00003520 con 0x7f0b00004d40 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b1d800640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007400 tx=0x7f0b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8b2acd75223bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10032070 con 0x7f0b00004d40 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1000b390 con 0x7f0b00004d40 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b17400640 1 -- 172.21.15.106:0/1074849383 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000bda0 con 0x7f0b00004d40 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b1f800640 1 -- 172.21.15.106:0/1074849383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 msgr2=0x7f0b000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1074849383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007400 tx=0x7f0b00005830 comp rx=0 tx=0).stop 2024-04-26T21:47:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b1d800640 1 -- 172.21.15.106:0/1074849383 reap_dead start 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.025+0000 7f0b1f800640 1 -- 172.21.15.106:0/1074849383 shutdown_connections 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.026+0000 7f0b1f800640 1 -- 172.21.15.106:0/1074849383 >> 172.21.15.106:0/1074849383 conn(0x7f0b180b2d50 msgr2=0x7f0b180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.026+0000 7f0b1f800640 1 -- 172.21.15.106:0/1074849383 shutdown_connections 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.026+0000 7f0b1f800640 1 -- 172.21.15.106:0/1074849383 wait complete. 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.027+0000 7f0b1f800640 1 Processor -- start 2024-04-26T21:47:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.027+0000 7f0b1f800640 1 -- start start 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b181524f0 0x7f0b180ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b180b4a00 0x7f0b18153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1815c600 con 0x7f0b00004d40 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180af0f0 con 0x7f0b180b4a00 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.028+0000 7f0b1f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1815c780 con 0x7f0b181524f0 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55352/0 (socket says 172.21.15.106:55352) 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 -- 172.21.15.106:0/1711103062 learned_addr learned my addr 172.21.15.106:0/1711103062 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 -- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b181524f0 msgr2=0x7f0b180ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b181524f0 0x7f0b180ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 -- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b180b4a00 msgr2=0x7f0b18153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b180b4a00 0x7f0b18153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 -- 172.21.15.106:0/1711103062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b18156370 con 0x7f0b00004d40 2024-04-26T21:47:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.029+0000 7f0b1ce00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00f5b0 tx=0x7f0b0c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.030+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c019020 con 0x7f0b00004d40 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.030+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c014870 con 0x7f0b00004d40 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.030+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b18152ab0 con 0x7f0b00004d40 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.030+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c014c60 con 0x7f0b00004d40 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.030+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b18153070 con 0x7f0b00004d40 2024-04-26T21:47:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.031+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b18069540 con 0x7f0b00004d40 2024-04-26T21:47:47.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.032+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0b0c01c020 con 0x7f0b00004d40 2024-04-26T21:47:47.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.033+0000 7f0b14c00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 0x7f0af806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.033+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0b0c08c650 con 0x7f0b00004d40 2024-04-26T21:47:47.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.033+0000 7f0b17e00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 0x7f0af806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.034+0000 7f0b17e00640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 0x7f0af806ef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b04001050 tx=0x7f0b04000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.037+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b0c023020 con 0x7f0b00004d40 2024-04-26T21:47:47.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.275+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b1806fdc0 con 0x7f0b00004d40 2024-04-26T21:47:47.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.276+0000 7f0b14c00640 1 -- 172.21.15.106:0/1711103062 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f0b0c026350 con 0x7f0b00004d40 2024-04-26T21:47:47.277 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:47.277 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:14.908611+0000","modified":"2024-04-26T21:47:17.987975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34541":{"gid":34541,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6835/107496404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":107496404},{"type":"v1","addr":"172.21.15.204:6835","nonce":107496404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34541,"qdb_cluster":[34541]},"id":2}]} 2024-04-26T21:47:47.278 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-26T21:47:47.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 msgr2=0x7f0af806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 0x7f0af806ef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b04001050 tx=0x7f0b04000f80 comp rx=0 tx=0).stop 2024-04-26T21:47:47.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 msgr2=0x7f0b18155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00f5b0 tx=0x7f0b0c00fa70 comp rx=0 tx=0).stop 2024-04-26T21:47:47.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 shutdown_connections 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b180b4a00 0x7f0b18153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0af806cad0 0x7f0af806ef90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b181524f0 0x7f0b180ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 --2- 172.21.15.106:0/1711103062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b00004d40 0x7f0b18155e30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 >> 172.21.15.106:0/1711103062 conn(0x7f0b180b2d50 msgr2=0x7f0b1815db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 shutdown_connections 2024-04-26T21:47:47.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.279+0000 7f0b1f800640 1 -- 172.21.15.106:0/1711103062 wait complete. 2024-04-26T21:47:47.297 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:47.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.485+0000 7f25eec00640 1 Processor -- start 2024-04-26T21:47:47.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.485+0000 7f25eec00640 1 -- start start 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25eec00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e815cb70 0x7f25e806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25eec00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e80ae850 con 0x7f25e815cfd0 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25eec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e80aeb30 con 0x7f25e806c770 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25eec00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f25e80af460 con 0x7f25e815cb70 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25ecc00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25e815cfd0 0x7f25e806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35756/0 (socket says 172.21.15.106:35756) 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25e7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25e815cb70 0x7f25e806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.487+0000 7f25ecc00640 1 -- 172.21.15.106:0/279791318 learned_addr learned my addr 172.21.15.106:0/279791318 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:47.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.488+0000 7f25e6a00640 1 -- 172.21.15.106:0/279791318 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902775648 0 0) 0x7f25e80ae850 con 0x7f25e815cfd0 2024-04-26T21:47:47.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.488+0000 7f25e6a00640 1 -- 172.21.15.106:0/279791318 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25cc003520 con 0x7f25e815cfd0 2024-04-26T21:47:47.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.488+0000 7f25e7400640 1 -- 172.21.15.106:0/279791318 >> v1:172.21.15.204:6789/0 conn(0x7f25e806c770 legacy=0x7f25e806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.488+0000 7f25e7400640 1 -- 172.21.15.106:0/279791318 >> v1:172.21.15.106:6789/0 conn(0x7f25e815cfd0 legacy=0x7f25e806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.488+0000 7f25e7400640 1 -- 172.21.15.106:0/279791318 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e80aee10 con 0x7f25e815cb70 2024-04-26T21:47:47.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.489+0000 7f25e7400640 1 --2- 172.21.15.106:0/279791318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e806ee60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f25e80af5e0 tx=0x7f25d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cbb82ada21f2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.489+0000 7f25e6a00640 1 -- 172.21.15.106:0/279791318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4037070 con 0x7f25e815cb70 2024-04-26T21:47:47.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.489+0000 7f25e6a00640 1 -- 172.21.15.106:0/279791318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d4032440 con 0x7f25e815cb70 2024-04-26T21:47:47.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.489+0000 7f25e6a00640 1 -- 172.21.15.106:0/279791318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4032810 con 0x7f25e815cb70 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 -- 172.21.15.106:0/279791318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 msgr2=0x7f25e806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 --2- 172.21.15.106:0/279791318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e806ee60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f25e80af5e0 tx=0x7f25d402a210 comp rx=0 tx=0).stop 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 -- 172.21.15.106:0/279791318 shutdown_connections 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 --2- 172.21.15.106:0/279791318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e815cb70 0x7f25e806ee60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 -- 172.21.15.106:0/279791318 >> 172.21.15.106:0/279791318 conn(0x7f25e80b2d50 msgr2=0x7f25e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 -- 172.21.15.106:0/279791318 shutdown_connections 2024-04-26T21:47:47.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.490+0000 7f25eec00640 1 -- 172.21.15.106:0/279791318 wait complete. 2024-04-26T21:47:47.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.492+0000 7f25eec00640 1 Processor -- start 2024-04-26T21:47:47.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.492+0000 7f25eec00640 1 -- start start 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 0x7f25e81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 0x7f25e81f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25e815c780 con 0x7f25e806c770 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25e815c600 con 0x7f25e815cb70 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25eec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25e80af8f0 con 0x7f25e815cfd0 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25e7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25e7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55372/0 (socket says 172.21.15.106:55372) 2024-04-26T21:47:47.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25e7e00640 1 -- 172.21.15.106:0/347664314 learned_addr learned my addr 172.21.15.106:0/347664314 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25e7400640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 0x7f25e81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25e7e00640 1 -- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 msgr2=0x7f25e81f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.493+0000 7f25ecc00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 0x7f25e81f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7e00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 0x7f25e81f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7e00640 1 -- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 msgr2=0x7f25e81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7e00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 0x7f25e81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7e00640 1 -- 172.21.15.106:0/347664314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8159d30 con 0x7f25e806c770 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7400640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 0x7f25e81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:47.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25ecc00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 0x7f25e81f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25e7e00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c3c0 tx=0x7f25d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80052d0 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8005490 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8007260 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e815a010 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.494+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e815a5d0 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.495+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e8069540 con 0x7f25e806c770 2024-04-26T21:47:47.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.496+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f25d8007400 con 0x7f25e806c770 2024-04-26T21:47:47.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.498+0000 7f25dfe00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 0x7f25c806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.498+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f25d808b370 con 0x7f25e806c770 2024-04-26T21:47:47.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.498+0000 7f25e7400640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 0x7f25c806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.499+0000 7f25e7400640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 0x7f25c806ef60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25d4034000 tx=0x7f25d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.503+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25d8059510 con 0x7f25e806c770 2024-04-26T21:47:47.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.710+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25e815a7a0 con 0x7f25e806c770 2024-04-26T21:47:47.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.713+0000 7f25dfe00640 1 -- 172.21.15.106:0/347664314 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16681 (secure 0 0 0) 0x7f25d8058e10 con 0x7f25e806c770 2024-04-26T21:47:47.714 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:47.715 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:16.482281+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:12.735544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:13.898739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 msgr2=0x7f25c806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 0x7f25c806ef60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25d4034000 tx=0x7f25d402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 msgr2=0x7f25e809c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c3c0 tx=0x7f25d800c880 comp rx=0 tx=0).stop 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 shutdown_connections 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f25c806caa0 0x7f25c806ef60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25e815cfd0 0x7f25e81f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25e815cb70 0x7f25e81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 --2- 172.21.15.106:0/347664314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25e806c770 0x7f25e809c270 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 >> 172.21.15.106:0/347664314 conn(0x7f25e80b2d50 msgr2=0x7f25e806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 shutdown_connections 2024-04-26T21:47:47.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.716+0000 7f25eec00640 1 -- 172.21.15.106:0/347664314 wait complete. 2024-04-26T21:47:47.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:47:47.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.939+0000 7f3f42800640 1 Processor -- start 2024-04-26T21:47:47.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.940+0000 7f3f42800640 1 -- start start 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f42800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f3c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f42800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0af9a0 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f42800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0afc80 con 0x7f3f3c0cd650 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f42800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c0ae2c0 con 0x7f3f3c06ce80 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f3b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f3c15cdf0 0x7f3f3c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54692/0 (socket says 172.21.15.106:54692) 2024-04-26T21:47:47.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f3b400640 1 -- 172.21.15.106:0/1642124189 learned_addr learned my addr 172.21.15.106:0/1642124189 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f3aa00640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f3c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.941+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159442481 0 0) 0x7f3f3c0af9a0 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f20003520 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1002011888 0 0) 0x7f3f20003520 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 msgr2=0x7f3f3c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f3c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 >> v1:172.21.15.204:6789/0 conn(0x7f3f3c0cd650 legacy=0x7f3f3c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0aff60 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f3f340019f0 con 0x7f3f3c15cdf0 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3aa00640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f3c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 >> v1:172.21.15.106:6789/0 conn(0x7f3f3c15cdf0 legacy=0x7f3f3c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f200039e0 0x7f3f20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f20004d40 0x7f3f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2000bb50 con 0x7f3f200039e0 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2000bcd0 con 0x7f3f20004d40 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2000bea0 con 0x7f3f3c06ce80 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.942+0000 7f3f3b400640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f200039e0 0x7f3f20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3b400640 1 -- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 msgr2=0x7f3f20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3b400640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f3c06ce80 0x7f3f20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3b400640 1 -- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f20004d40 msgr2=0x7f3f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3be00640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f20004d40 0x7f3f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3b400640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f20004d40 0x7f3f200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3b400640 1 -- 172.21.15.106:0/1642124189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f20003520 con 0x7f3f200039e0 2024-04-26T21:47:47.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.943+0000 7f3f3be00640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f20004d40 0x7f3f200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.944+0000 7f3f3b400640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f200039e0 0x7f3f20003e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400a770 tx=0x7f3f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633c8ac4aaa5aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.944+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400a390 con 0x7f3f200039e0 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.944+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3402f070 con 0x7f3f200039e0 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.944+0000 7f3f3a000640 1 -- 172.21.15.106:0/1642124189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3402bba0 con 0x7f3f200039e0 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 -- 172.21.15.106:0/1642124189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f200039e0 msgr2=0x7f3f20003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 --2- 172.21.15.106:0/1642124189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f200039e0 0x7f3f20003e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400a770 tx=0x7f3f20005830 comp rx=0 tx=0).stop 2024-04-26T21:47:47.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f3be00640 1 -- 172.21.15.106:0/1642124189 reap_dead start 2024-04-26T21:47:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 -- 172.21.15.106:0/1642124189 shutdown_connections 2024-04-26T21:47:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 -- 172.21.15.106:0/1642124189 >> 172.21.15.106:0/1642124189 conn(0x7f3f3c0b2d50 msgr2=0x7f3f3c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 -- 172.21.15.106:0/1642124189 shutdown_connections 2024-04-26T21:47:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.945+0000 7f3f42800640 1 -- 172.21.15.106:0/1642124189 wait complete. 2024-04-26T21:47:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.946+0000 7f3f42800640 1 Processor -- start 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.947+0000 7f3f42800640 1 -- start start 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f200039e0 0x7f3f3c09dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c15f630 0x7f3f3c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d1d0 con 0x7f3f3c06f190 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c0b05b0 con 0x7f3f3c15f630 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f42800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d350 con 0x7f3f200039e0 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39540/0 (socket says 172.21.15.106:39540) 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 -- 172.21.15.106:0/3536447450 learned_addr learned my addr 172.21.15.106:0/3536447450 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 -- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f200039e0 msgr2=0x7f3f3c09dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3b400640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f200039e0 0x7f3f3c09dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3be00640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c15f630 0x7f3f3c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f200039e0 0x7f3f3c09dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 -- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c15f630 msgr2=0x7f3f3c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c15f630 0x7f3f3c15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.948+0000 7f3f3aa00640 1 -- 172.21.15.106:0/3536447450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1f4f60 con 0x7f3f3c06f190 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f3aa00640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e810 tx=0x7f3f2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24017070 con 0x7f3f3c06f190 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2400c6d0 con 0x7f3f3c06f190 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400f040 con 0x7f3f3c06f190 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c1f5240 con 0x7f3f3c06f190 2024-04-26T21:47:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.949+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c1f5800 con 0x7f3f3c06f190 2024-04-26T21:47:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.950+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c069540 con 0x7f3f3c06f190 2024-04-26T21:47:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.951+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3f2401a070 con 0x7f3f3c06f190 2024-04-26T21:47:47.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.952+0000 7f3f33400640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 0x7f3f1806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.952+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3f2408ba10 con 0x7f3f3c06f190 2024-04-26T21:47:47.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.952+0000 7f3f3b400640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 0x7f3f1806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.953+0000 7f3f3b400640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 0x7f3f1806ef90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f34033000 tx=0x7f3f34007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:47.956+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f24059aa0 con 0x7f3f3c06f190 2024-04-26T21:47:48.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.195+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3f3c155e10 con 0x7f3f3c06f190 2024-04-26T21:47:48.197 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-26T21:47:48.196+0000 7fca30c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:47:48.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.200+0000 7f3f33400640 1 -- 172.21.15.106:0/3536447450 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f3f240593a0 con 0x7f3f3c06f190 2024-04-26T21:47:48.201 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:47:48.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 msgr2=0x7f3f1806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 0x7f3f1806ef90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f34033000 tx=0x7f3f34007000 comp rx=0 tx=0).stop 2024-04-26T21:47:48.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 msgr2=0x7f3f3c06f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e810 tx=0x7f3f2400ecd0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 shutdown_connections 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c15f630 0x7f3f3c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c06f190 0x7f3f3c06f610 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1806cad0 0x7f3f1806ef90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 --2- 172.21.15.106:0/3536447450 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f200039e0 0x7f3f3c09dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 >> 172.21.15.106:0/3536447450 conn(0x7f3f3c0b2d50 msgr2=0x7f3f3c0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 shutdown_connections 2024-04-26T21:47:48.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.203+0000 7f3f42800640 1 -- 172.21.15.106:0/3536447450 wait complete. 2024-04-26T21:47:48.222 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:47:48.333 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-26T21:47:48.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.408+0000 7fc7f3e00640 1 Processor -- start 2024-04-26T21:47:48.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.408+0000 7fc7f3e00640 1 -- start start 2024-04-26T21:47:48.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.409+0000 7fc7f3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 0x7fc7ec06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.409+0000 7fc7f3e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7ec0b5ca0 0x7fc7ec06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f3e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ec0af800 con 0x7fc7ec0add80 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f3e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ec0ae340 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec0afaa0 con 0x7fc7ec15d100 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f0a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7ec0b5ca0 0x7fc7ec06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f1e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc7ec0add80 0x7fc7ec072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54694/0 (socket says 172.21.15.106:54694) 2024-04-26T21:47:48.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f1e00640 1 -- 172.21.15.106:0/2031611297 learned_addr learned my addr 172.21.15.106:0/2031611297 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.410+0000 7fc7f1400640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 0x7fc7ec06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7f0a00640 1 -- 172.21.15.106:0/2031611297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 msgr2=0x7fc7ec06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7f0a00640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 0x7fc7ec06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7f0a00640 1 -- 172.21.15.106:0/2031611297 >> v1:172.21.15.106:6789/0 conn(0x7fc7ec0add80 legacy=0x7fc7ec072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7f0a00640 1 -- 172.21.15.106:0/2031611297 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec0afd80 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7f1400640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 0x7fc7ec06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:48.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.411+0000 7fc7ebe00640 1 -- 172.21.15.106:0/2031611297 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188611807 0 0) 0x7fc7ec0af800 con 0x7fc7ec0add80 2024-04-26T21:47:48.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.412+0000 7fc7f0a00640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0b5ca0 0x7fc7ec06ccb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80097a0 tx=0x7fc7d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc20bbc06194a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.412+0000 7fc7ebe00640 1 -- 172.21.15.106:0/2031611297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8037070 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.412+0000 7fc7ebe00640 1 -- 172.21.15.106:0/2031611297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d802ae50 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.412+0000 7fc7ebe00640 1 -- 172.21.15.106:0/2031611297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8032660 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2031611297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0b5ca0 msgr2=0x7fc7ec06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0b5ca0 0x7fc7ec06ccb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80097a0 tx=0x7fc7d802a100 comp rx=0 tx=0).stop 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2031611297 shutdown_connections 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0b5ca0 0x7fc7ec06ccb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2031611297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec15d100 0x7fc7ec06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2031611297 >> 172.21.15.106:0/2031611297 conn(0x7fc7ec0b2d50 msgr2=0x7fc7ec0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:48.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2031611297 shutdown_connections 2024-04-26T21:47:48.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.413+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2031611297 wait complete. 2024-04-26T21:47:48.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.415+0000 7fc7f3e00640 1 Processor -- start 2024-04-26T21:47:48.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.415+0000 7fc7f3e00640 1 -- start start 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec0b5ca0 0x7fc7ec1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 0x7fc7ec09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec09ee50 con 0x7fc7ec0add80 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec0b0340 con 0x7fc7ec15d100 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec160b70 con 0x7fc7ec0b5ca0 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39562/0 (socket says 172.21.15.106:39562) 2024-04-26T21:47:48.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 -- 172.21.15.106:0/2434266560 learned_addr learned my addr 172.21.15.106:0/2434266560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:48.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 0x7fc7ec09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 -- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec0b5ca0 msgr2=0x7fc7ec1f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec0b5ca0 0x7fc7ec1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 -- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 msgr2=0x7fc7ec09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.416+0000 7fc7f1400640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 0x7fc7ec09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7f1400640 1 -- 172.21.15.106:0/2434266560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec159f20 con 0x7fc7ec0add80 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7f1e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 0x7fc7ec09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7f1400640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e730 tx=0x7fc7e000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000f040 con 0x7fc7ec0add80 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e0004230 con 0x7fc7ec0add80 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000ee40 con 0x7fc7ec0add80 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec15a200 con 0x7fc7ec0add80 2024-04-26T21:47:48.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.417+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec15a7c0 con 0x7fc7ec0add80 2024-04-26T21:47:48.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.418+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec069650 con 0x7fc7ec0add80 2024-04-26T21:47:48.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.420+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc7e000c6d0 con 0x7fc7ec0add80 2024-04-26T21:47:48.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.420+0000 7fc7e9600640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 0x7fc7cc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.421+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc7e008be20 con 0x7fc7ec0add80 2024-04-26T21:47:48.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.421+0000 7fc7f0a00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 0x7fc7cc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.421+0000 7fc7f0a00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 0x7fc7cc06ef60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8034000 tx=0x7fc7d802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.425+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7e0017020 con 0x7fc7ec0add80 2024-04-26T21:47:48.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:48.667+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc7ec073d00 con 0x7fc7ec0add80 2024-04-26T21:47:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.203+0000 7fc7e9600640 1 -- 172.21.15.106:0/2434266560 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fc7e0059dc0 con 0x7fc7ec0add80 2024-04-26T21:47:49.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.206+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 msgr2=0x7fc7cc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.206+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 0x7fc7cc06ef60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8034000 tx=0x7fc7d802c000 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.206+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 msgr2=0x7fc7ec160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.206+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e730 tx=0x7fc7e000ebf0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 shutdown_connections 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7ec15d100 0x7fc7ec09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc7cc06caa0 0x7fc7cc06ef60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7ec0b5ca0 0x7fc7ec1f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 --2- 172.21.15.106:0/2434266560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7ec0add80 0x7fc7ec160110 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 >> 172.21.15.106:0/2434266560 conn(0x7fc7ec0b2d50 msgr2=0x7fc7ec0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 shutdown_connections 2024-04-26T21:47:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.207+0000 7fc7f3e00640 1 -- 172.21.15.106:0/2434266560 wait complete. 2024-04-26T21:47:49.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:47:49.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.436+0000 7f8128a00640 1 Processor -- start 2024-04-26T21:47:49.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.436+0000 7f8128a00640 1 -- start start 2024-04-26T21:47:49.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f8128a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f81241459b0 0x7f81240cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f8128a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8124072a20 con 0x7f8124071e60 2024-04-26T21:47:49.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f8128a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81240cf210 con 0x7f81240d1650 2024-04-26T21:47:49.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f8128a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f81240b5be0 con 0x7f81241459b0 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f811be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8124071e60 0x7f81240d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54708/0 (socket says 172.21.15.106:54708) 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.438+0000 7f811be00640 1 -- 172.21.15.106:0/3149893068 learned_addr learned my addr 172.21.15.106:0/3149893068 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8123400640 1 --2- 172.21.15.106:0/3149893068 >> v2:172.21.15.204:3300/0 conn(0x7f81241459b0 0x7f81240cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762835685 0 0) 0x7f8124072a20 con 0x7f8124071e60 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8108003520 con 0x7f8124071e60 2024-04-26T21:47:49.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 787847941 0 0) 0x7f8108003520 con 0x7f8124071e60 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 >> v2:172.21.15.204:3300/0 conn(0x7f81241459b0 msgr2=0x7f81240cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 --2- 172.21.15.106:0/3149893068 >> v2:172.21.15.204:3300/0 conn(0x7f81241459b0 0x7f81240cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 >> v1:172.21.15.204:6789/0 conn(0x7f81240d1650 legacy=0x7f81240b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81240cf4f0 con 0x7f8124071e60 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f810c001670 con 0x7f8124071e60 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 >> v1:172.21.15.106:6789/0 conn(0x7f8124071e60 legacy=0x7f81240d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080039b0 0x7f8108003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8108004800 0x7f8108004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 0x7f810800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.439+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f810800c030 con 0x7f81080051c0 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f810800c1e0 con 0x7f81080039b0 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f810800c3e0 con 0x7f8108004800 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 0x7f810800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123400640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080039b0 0x7f8108003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f811be00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8108004800 0x7f8108004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 -- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8108004800 msgr2=0x7f8108004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8108004800 0x7f8108004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 -- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080039b0 msgr2=0x7f8108003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080039b0 0x7f8108003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.440+0000 7f8123e00640 1 -- 172.21.15.106:0/3149893068 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81240cf4f0 con 0x7f81080051c0 2024-04-26T21:47:49.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.441+0000 7f8123e00640 1 -- 172.21.15.106:0/3149893068 reap_dead start 2024-04-26T21:47:49.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.442+0000 7f8123e00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 0x7f810800b9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f811c001e20 tx=0x7f8108005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2313d7ed4b6d9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.442+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c001a70 con 0x7f81080051c0 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.442+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c02ae40 con 0x7f81080051c0 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.442+0000 7f8122a00640 1 -- 172.21.15.106:0/3149893068 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c003bb0 con 0x7f81080051c0 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 -- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 msgr2=0x7f810800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 0x7f810800b9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f811c001e20 tx=0x7f8108005830 comp rx=0 tx=0).stop 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 -- 172.21.15.106:0/3149893068 shutdown_connections 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 --2- 172.21.15.106:0/3149893068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81080051c0 0x7f810800b9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 -- 172.21.15.106:0/3149893068 >> 172.21.15.106:0/3149893068 conn(0x7f812406c770 msgr2=0x7f812406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 -- 172.21.15.106:0/3149893068 shutdown_connections 2024-04-26T21:47:49.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.443+0000 7f8128a00640 1 -- 172.21.15.106:0/3149893068 wait complete. 2024-04-26T21:47:49.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.445+0000 7f8128a00640 1 Processor -- start 2024-04-26T21:47:49.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.445+0000 7f8128a00640 1 -- start start 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.446+0000 7f8128a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080051c0 0x7f81240b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.446+0000 7f8128a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81240ac880 0x7f81240acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8128a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8128a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8124145d90 con 0x7f81241f4b80 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8128a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8124072d00 con 0x7f81080051c0 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8128a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8124145f10 con 0x7f81240ac880 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39586/0 (socket says 172.21.15.106:39586) 2024-04-26T21:47:49.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 -- 172.21.15.106:0/2179333124 learned_addr learned my addr 172.21.15.106:0/2179333124 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:49.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 -- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81240ac880 msgr2=0x7f81240acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81240ac880 0x7f81240acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 -- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080051c0 msgr2=0x7f81240b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080051c0 0x7f81240b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.447+0000 7f8123e00640 1 -- 172.21.15.106:0/2179333124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241f7ed0 con 0x7f81241f4b80 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f8123e00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f811c001f40 tx=0x7f811c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c007db0 con 0x7f81241f4b80 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c006d60 con 0x7f81241f4b80 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81240ad2e0 con 0x7f81241f4b80 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c005720 con 0x7f81241f4b80 2024-04-26T21:47:49.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.448+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241f8470 con 0x7f81241f4b80 2024-04-26T21:47:49.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.449+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81240693c0 con 0x7f81241f4b80 2024-04-26T21:47:49.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.450+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f811c007030 con 0x7f81241f4b80 2024-04-26T21:47:49.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.451+0000 7f811b400640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 0x7f810006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.451+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f811c0ab650 con 0x7f81241f4b80 2024-04-26T21:47:49.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.451+0000 7f8123400640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 0x7f810006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.452+0000 7f8123400640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 0x7f810006ef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8114009a40 tx=0x7f8114009320 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.455+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f811c043060 con 0x7f81241f4b80 2024-04-26T21:47:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:49.654+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f812415d030 con 0x7f81241f4b80 2024-04-26T21:47:50.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.207+0000 7f811b400640 1 -- 172.21.15.106:0/2179333124 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f811c039070 con 0x7f81241f4b80 2024-04-26T21:47:50.208 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:47:50.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 msgr2=0x7f810006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 0x7f810006ef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8114009a40 tx=0x7f8114009320 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 msgr2=0x7f81241f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f811c001f40 tx=0x7f811c0014a0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 shutdown_connections 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81241f4b80 0x7f81241f4fe0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f810006caa0 0x7f810006ef60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81240ac880 0x7f81240acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 --2- 172.21.15.106:0/2179333124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81080051c0 0x7f81240b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.211+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 >> 172.21.15.106:0/2179333124 conn(0x7f812406c770 msgr2=0x7f81240d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.212+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 shutdown_connections 2024-04-26T21:47:50.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.212+0000 7f8128a00640 1 -- 172.21.15.106:0/2179333124 wait complete. 2024-04-26T21:47:50.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:47:50.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.433+0000 7fda86e00640 1 Processor -- start 2024-04-26T21:47:50.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.433+0000 7fda86e00640 1 -- start start 2024-04-26T21:47:50.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.435+0000 7fda86e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fda80163af0 0x7fda800b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.435+0000 7fda86e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda80163f60 con 0x7fda80161020 2024-04-26T21:47:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.435+0000 7fda86e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda801641f0 con 0x7fda80073710 2024-04-26T21:47:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.435+0000 7fda86e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fda80164a90 con 0x7fda80163af0 2024-04-26T21:47:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fda80161020 0x7fda8006fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54718/0 (socket says 172.21.15.106:54718) 2024-04-26T21:47:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7fe00640 1 -- 172.21.15.106:0/3803779097 learned_addr learned my addr 172.21.15.106:0/3803779097 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461710205 0 0) 0x7fda80163f60 con 0x7fda80161020 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda64003520 con 0x7fda80161020 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda77e00640 1 --2- 172.21.15.106:0/3803779097 >> v2:172.21.15.204:3300/0 conn(0x7fda80163af0 0x7fda800b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2567973938 0 0) 0x7fda64003520 con 0x7fda80161020 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 >> v2:172.21.15.204:3300/0 conn(0x7fda80163af0 msgr2=0x7fda800b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 --2- 172.21.15.106:0/3803779097 >> v2:172.21.15.204:3300/0 conn(0x7fda80163af0 0x7fda800b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 >> v1:172.21.15.204:6789/0 conn(0x7fda80073710 legacy=0x7fda800727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801644d0 con 0x7fda80161020 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.436+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fda70001670 con 0x7fda80161020 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 >> v1:172.21.15.106:6789/0 conn(0x7fda80161020 legacy=0x7fda8006fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda64003a00 0x7fda64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda64004800 0x7fda64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 0x7fda6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda80164a90 con 0x7fda640053c0 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda6400c280 con 0x7fda64003a00 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fda6400c480 con 0x7fda64004800 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda84e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 0x7fda6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda7fe00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda64003a00 0x7fda64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.437+0000 7fda77e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda64004800 0x7fda64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.438+0000 7fda84e00640 1 -- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda64004800 msgr2=0x7fda64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.438+0000 7fda84e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda64004800 0x7fda64004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.438+0000 7fda84e00640 1 -- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda64003a00 msgr2=0x7fda64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.438+0000 7fda84e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda64003a00 0x7fda64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.438+0000 7fda84e00640 1 -- 172.21.15.106:0/3803779097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda64003520 con 0x7fda640053c0 2024-04-26T21:47:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.439+0000 7fda84e00640 1 -- 172.21.15.106:0/3803779097 reap_dead start 2024-04-26T21:47:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.439+0000 7fda84e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 0x7fda6400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda780044b0 tx=0x7fda64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9d78a47043a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.439+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78004080 con 0x7fda640053c0 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.439+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7802ada0 con 0x7fda640053c0 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.439+0000 7fda7f400640 1 -- 172.21.15.106:0/3803779097 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78006d90 con 0x7fda640053c0 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 -- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 msgr2=0x7fda6400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 0x7fda6400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda780044b0 tx=0x7fda64005830 comp rx=0 tx=0).stop 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 -- 172.21.15.106:0/3803779097 shutdown_connections 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 --2- 172.21.15.106:0/3803779097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda640053c0 0x7fda6400b9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 -- 172.21.15.106:0/3803779097 >> 172.21.15.106:0/3803779097 conn(0x7fda800d0c40 msgr2=0x7fda800d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 -- 172.21.15.106:0/3803779097 shutdown_connections 2024-04-26T21:47:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.440+0000 7fda86e00640 1 -- 172.21.15.106:0/3803779097 wait complete. 2024-04-26T21:47:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.442+0000 7fda86e00640 1 Processor -- start 2024-04-26T21:47:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.442+0000 7fda86e00640 1 -- start start 2024-04-26T21:47:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda640053c0 0x7fda801ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda8009e020 0x7fda8009e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda80161400 con 0x7fda801ec760 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda80073b60 con 0x7fda640053c0 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda86e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fda8009e9e0 con 0x7fda8009e020 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda77e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda77e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39606/0 (socket says 172.21.15.106:39606) 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.443+0000 7fda77e00640 1 -- 172.21.15.106:0/2470991266 learned_addr learned my addr 172.21.15.106:0/2470991266 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda7fe00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda640053c0 0x7fda801ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda84e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda8009e020 0x7fda8009e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 -- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda8009e020 msgr2=0x7fda8009e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda8009e020 0x7fda8009e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 -- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda640053c0 msgr2=0x7fda801ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda640053c0 0x7fda801ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 -- 172.21.15.106:0/2470991266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801ecee0 con 0x7fda801ec760 2024-04-26T21:47:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda77e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fda6800e7e0 tx=0x7fda6800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68017070 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6800c6d0 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6800f040 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.444+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda801ed1c0 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.445+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8009bf80 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.446+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda80069650 con 0x7fda801ec760 2024-04-26T21:47:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.447+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fda6801a070 con 0x7fda801ec760 2024-04-26T21:47:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.448+0000 7fda7cc00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 0x7fda5c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.448+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7fda6808bb10 con 0x7fda801ec760 2024-04-26T21:47:50.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.448+0000 7fda7fe00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 0x7fda5c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.449+0000 7fda7fe00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 0x7fda5c06ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.452+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda68059d10 con 0x7fda801ec760 2024-04-26T21:47:50.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:50.666+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda800d0a20 con 0x7fda801ec760 2024-04-26T21:47:51.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.211+0000 7fda7cc00640 1 -- 172.21.15.106:0/2470991266 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7fda68059610 con 0x7fda801ec760 2024-04-26T21:47:51.212 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-26T21:47:51.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 msgr2=0x7fda5c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 0x7fda5c06ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70009040 comp rx=0 tx=0).stop 2024-04-26T21:47:51.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 msgr2=0x7fda80152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fda6800e7e0 tx=0x7fda6800eca0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 shutdown_connections 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fda5c06caa0 0x7fda5c06ef60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda8009e020 0x7fda8009e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda801ec760 0x7fda80152dd0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 --2- 172.21.15.106:0/2470991266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda640053c0 0x7fda801ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 >> 172.21.15.106:0/2470991266 conn(0x7fda800d0c40 msgr2=0x7fda80166660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.214+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 shutdown_connections 2024-04-26T21:47:51.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.215+0000 7fda86e00640 1 -- 172.21.15.106:0/2470991266 wait complete. 2024-04-26T21:47:51.242 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:47:51.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.442+0000 7faf28200640 1 Processor -- start 2024-04-26T21:47:51.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.442+0000 7faf28200640 1 -- start start 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf28200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faf2006ce80 0x7faf200b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf28200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf200af800 con 0x7faf200b5ca0 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf28200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf200ae340 con 0x7faf2015d100 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf28200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7faf200afaa0 con 0x7faf2006ce80 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf24e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faf2006ce80 0x7faf200b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf26200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7faf200b5ca0 0x7faf200b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54728/0 (socket says 172.21.15.106:54728) 2024-04-26T21:47:51.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.444+0000 7faf26200640 1 -- 172.21.15.106:0/2412881230 learned_addr learned my addr 172.21.15.106:0/2412881230 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.445+0000 7faf24e00640 1 -- 172.21.15.106:0/2412881230 >> v1:172.21.15.204:6789/0 conn(0x7faf2015d100 legacy=0x7faf2006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.445+0000 7faf24e00640 1 -- 172.21.15.106:0/2412881230 >> v1:172.21.15.106:6789/0 conn(0x7faf200b5ca0 legacy=0x7faf200b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.445+0000 7faf1fe00640 1 -- 172.21.15.106:0/2412881230 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343302759 0 0) 0x7faf200af800 con 0x7faf200b5ca0 2024-04-26T21:47:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.445+0000 7faf24e00640 1 -- 172.21.15.106:0/2412881230 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf200afd80 con 0x7faf2006ce80 2024-04-26T21:47:51.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.446+0000 7faf24e00640 1 --2- 172.21.15.106:0/2412881230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf2006ce80 0x7faf200b3c30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0097a0 tx=0x7faf0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e73d8b75564557f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.446+0000 7faf1fe00640 1 -- 172.21.15.106:0/2412881230 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c037070 con 0x7faf2006ce80 2024-04-26T21:47:51.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.446+0000 7faf1fe00640 1 -- 172.21.15.106:0/2412881230 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c032440 con 0x7faf2006ce80 2024-04-26T21:47:51.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.446+0000 7faf1fe00640 1 -- 172.21.15.106:0/2412881230 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c032810 con 0x7faf2006ce80 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 -- 172.21.15.106:0/2412881230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf2006ce80 msgr2=0x7faf200b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 --2- 172.21.15.106:0/2412881230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf2006ce80 0x7faf200b3c30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0097a0 tx=0x7faf0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 -- 172.21.15.106:0/2412881230 shutdown_connections 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 --2- 172.21.15.106:0/2412881230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf2006ce80 0x7faf200b3c30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 -- 172.21.15.106:0/2412881230 >> 172.21.15.106:0/2412881230 conn(0x7faf200b2d50 msgr2=0x7faf200b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 -- 172.21.15.106:0/2412881230 shutdown_connections 2024-04-26T21:47:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.447+0000 7faf28200640 1 -- 172.21.15.106:0/2412881230 wait complete. 2024-04-26T21:47:51.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.449+0000 7faf28200640 1 Processor -- start 2024-04-26T21:47:51.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.449+0000 7faf28200640 1 -- start start 2024-04-26T21:47:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf2006ce80 0x7faf201e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf2015d100 0x7faf2006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faf2009ee50 con 0x7faf200b5ca0 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf200b0340 con 0x7faf2006ce80 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.450+0000 7faf28200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faf20160b70 con 0x7faf2015d100 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39626/0 (socket says 172.21.15.106:39626) 2024-04-26T21:47:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 -- 172.21.15.106:0/3653486944 learned_addr learned my addr 172.21.15.106:0/3653486944 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 -- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf2015d100 msgr2=0x7faf2006f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf2015d100 0x7faf2006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 -- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf2006ce80 msgr2=0x7faf201e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf2006ce80 0x7faf201e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 -- 172.21.15.106:0/3653486944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf201e6f90 con 0x7faf200b5ca0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.451+0000 7faf24e00640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7faf0c000c00 tx=0x7faf0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.452+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c037040 con 0x7faf200b5ca0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.452+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c042070 con 0x7faf200b5ca0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.452+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c030a70 con 0x7faf200b5ca0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.452+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf201e7270 con 0x7faf200b5ca0 2024-04-26T21:47:51.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.452+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201e7830 con 0x7faf200b5ca0 2024-04-26T21:47:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.453+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf20069650 con 0x7faf200b5ca0 2024-04-26T21:47:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.454+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7faf0c030c10 con 0x7faf200b5ca0 2024-04-26T21:47:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.455+0000 7faf1d600640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 0x7faf0006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.455+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7faf0c0ac720 con 0x7faf200b5ca0 2024-04-26T21:47:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.455+0000 7faf25800640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 0x7faf0006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.456+0000 7faf25800640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 0x7faf0006ef60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf14000f60 tx=0x7faf140079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.459+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf0c045360 con 0x7faf200b5ca0 2024-04-26T21:47:51.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.667+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7faf200b35f0 con 0x7faf200b5ca0 2024-04-26T21:47:51.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.672+0000 7faf1d600640 1 -- 172.21.15.106:0/3653486944 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7faf0c07ab30 con 0x7faf200b5ca0 2024-04-26T21:47:51.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 msgr2=0x7faf0006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 0x7faf0006ef60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf14000f60 tx=0x7faf140079b0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 msgr2=0x7faf201467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7faf0c000c00 tx=0x7faf0c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 shutdown_connections 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7faf0006caa0 0x7faf0006ef60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf2015d100 0x7faf2006f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf200b5ca0 0x7faf201467f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 --2- 172.21.15.106:0/3653486944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf2006ce80 0x7faf201e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 >> 172.21.15.106:0/3653486944 conn(0x7faf200b2d50 msgr2=0x7faf201569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 shutdown_connections 2024-04-26T21:47:51.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.675+0000 7faf28200640 1 -- 172.21.15.106:0/3653486944 wait complete. 2024-04-26T21:47:51.696 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-26T21:47:51.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.893+0000 7feba6400640 1 Processor -- start 2024-04-26T21:47:51.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.893+0000 7feba6400640 1 -- start start 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.894+0000 7feba6400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feba006c770 0x7feba006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.894+0000 7feba6400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feba015cb70 0x7feba0073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.894+0000 7feba6400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba006d090 con 0x7feba015cfd0 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.894+0000 7feba6400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feba00af460 con 0x7feba006c770 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.894+0000 7feba6400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feba00ae850 con 0x7feba015cb70 2024-04-26T21:47:51.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feba006c770 0x7feba006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feba015cfd0 0x7feba0073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54734/0 (socket says 172.21.15.106:54734) 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9fe00640 1 -- 172.21.15.106:0/2838846473 learned_addr learned my addr 172.21.15.106:0/2838846473 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9ea00640 1 --2- 172.21.15.106:0/2838846473 >> v2:172.21.15.204:3300/0 conn(0x7feba015cb70 0x7feba0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9f400640 1 -- 172.21.15.106:0/2838846473 >> v2:172.21.15.204:3300/0 conn(0x7feba015cb70 msgr2=0x7feba0073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9e000640 1 -- 172.21.15.106:0/2838846473 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064257846 0 0) 0x7feba006d090 con 0x7feba015cfd0 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9f400640 1 --2- 172.21.15.106:0/2838846473 >> v2:172.21.15.204:3300/0 conn(0x7feba015cb70 0x7feba0073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9f400640 1 -- 172.21.15.106:0/2838846473 >> v1:172.21.15.106:6789/0 conn(0x7feba015cfd0 legacy=0x7feba0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.895+0000 7feb9f400640 1 -- 172.21.15.106:0/2838846473 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00aeb30 con 0x7feba006c770 2024-04-26T21:47:51.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.896+0000 7feb9f400640 1 --2- 172.21.15.106:0/2838846473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba006c770 0x7feba006cb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feba00afec0 tx=0x7feb9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc413e9a4de7f674 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feb9e000640 1 -- 172.21.15.106:0/2838846473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90037070 con 0x7feba006c770 2024-04-26T21:47:51.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feb9e000640 1 -- 172.21.15.106:0/2838846473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb90032440 con 0x7feba006c770 2024-04-26T21:47:51.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feb9e000640 1 -- 172.21.15.106:0/2838846473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90032810 con 0x7feba006c770 2024-04-26T21:47:51.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 -- 172.21.15.106:0/2838846473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba006c770 msgr2=0x7feba006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 --2- 172.21.15.106:0/2838846473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba006c770 0x7feba006cb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feba00afec0 tx=0x7feb9002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 -- 172.21.15.106:0/2838846473 shutdown_connections 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 --2- 172.21.15.106:0/2838846473 >> v2:172.21.15.204:3300/0 conn(0x7feba015cb70 0x7feba0073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 --2- 172.21.15.106:0/2838846473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba006c770 0x7feba006cb50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 -- 172.21.15.106:0/2838846473 >> 172.21.15.106:0/2838846473 conn(0x7feba00b2d50 msgr2=0x7feba00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 -- 172.21.15.106:0/2838846473 shutdown_connections 2024-04-26T21:47:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.897+0000 7feba6400640 1 -- 172.21.15.106:0/2838846473 wait complete. 2024-04-26T21:47:51.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.899+0000 7feba6400640 1 Processor -- start 2024-04-26T21:47:51.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.899+0000 7feba6400640 1 -- start start 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba006c770 0x7feba00cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 0x7feba00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feba015c600 con 0x7feba015cfd0 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feba00af0f0 con 0x7feba015cb70 2024-04-26T21:47:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feba6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feba015c780 con 0x7feba006c770 2024-04-26T21:47:51.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feb9fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feb9fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39648/0 (socket says 172.21.15.106:39648) 2024-04-26T21:47:51.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.900+0000 7feb9fe00640 1 -- 172.21.15.106:0/2920076887 learned_addr learned my addr 172.21.15.106:0/2920076887 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9ea00640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 0x7feba00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 -- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba006c770 msgr2=0x7feba00cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9f400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba006c770 0x7feba00cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba006c770 0x7feba00cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 -- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 msgr2=0x7feba00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 0x7feba00b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 -- 172.21.15.106:0/2920076887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00d04d0 con 0x7feba015cfd0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9ea00640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 0x7feba00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb9fe00640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c700 tx=0x7feb9800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98018070 con 0x7feba015cfd0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9800ce10 con 0x7feba015cfd0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.901+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800a860 con 0x7feba015cfd0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.902+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba009c270 con 0x7feba015cfd0 2024-04-26T21:47:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.902+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba00cf760 con 0x7feba015cfd0 2024-04-26T21:47:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.903+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba0069540 con 0x7feba015cfd0 2024-04-26T21:47:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.904+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7feb9800aa00 con 0x7feba015cfd0 2024-04-26T21:47:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.904+0000 7feb97400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 0x7feb8006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.905+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7feb9808b1a0 con 0x7feba015cfd0 2024-04-26T21:47:51.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.905+0000 7feb9f400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 0x7feb8006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.905+0000 7feb9f400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 0x7feb8006ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feb90034000 tx=0x7feb9002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:51.909+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb98059640 con 0x7feba015cfd0 2024-04-26T21:47:52.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.187+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7feba00cd430 con 0x7feba015cfd0 2024-04-26T21:47:52.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.214+0000 7feb97400640 1 -- 172.21.15.106:0/2920076887 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v127) ==== 158+0+0 (secure 0 0 0) 0x7feb98058f40 con 0x7feba015cfd0 2024-04-26T21:47:52.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 msgr2=0x7feb8006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 0x7feb8006ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feb90034000 tx=0x7feb9002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:52.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 msgr2=0x7feba00b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c700 tx=0x7feb9800cbc0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 shutdown_connections 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feba015cfd0 0x7feba00b1f60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba015cb70 0x7feba00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feb8006caa0 0x7feb8006ef60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 --2- 172.21.15.106:0/2920076887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba006c770 0x7feba00cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 >> 172.21.15.106:0/2920076887 conn(0x7feba00b2d50 msgr2=0x7feba006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 shutdown_connections 2024-04-26T21:47:52.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.217+0000 7feba6400640 1 -- 172.21.15.106:0/2920076887 wait complete. 2024-04-26T21:47:52.238 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-04-26T21:47:52.238 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:47:52.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:52.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.436+0000 7f1409800640 1 Processor -- start 2024-04-26T21:47:52.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.436+0000 7f1409800640 1 -- start start 2024-04-26T21:47:52.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.438+0000 7f1409800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140406ce80 0x7f14040add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.438+0000 7f1409800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040af9a0 con 0x7f14040cd650 2024-04-26T21:47:52.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.438+0000 7f1409800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040afc80 con 0x7f140415cdf0 2024-04-26T21:47:52.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.438+0000 7f1409800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14040ae2c0 con 0x7f140406ce80 2024-04-26T21:47:52.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.439+0000 7f1403400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f14040cd650 0x7f14040b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54736/0 (socket says 172.21.15.106:54736) 2024-04-26T21:47:52.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.439+0000 7f1403400640 1 -- 172.21.15.106:0/4292409664 learned_addr learned my addr 172.21.15.106:0/4292409664 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.439+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140406ce80 0x7f14040add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.439+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913471541 0 0) 0x7f14040af9a0 con 0x7f14040cd650 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.439+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e8003520 con 0x7f14040cd650 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424351551 0 0) 0x7f13e8003520 con 0x7f14040cd650 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140406ce80 msgr2=0x7f14040add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140406ce80 0x7f14040add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 >> v1:172.21.15.204:6789/0 conn(0x7f140415cdf0 legacy=0x7f140406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040aff60 con 0x7f14040cd650 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f13fc0019f0 con 0x7f14040cd650 2024-04-26T21:47:52.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 >> v1:172.21.15.106:6789/0 conn(0x7f14040cd650 legacy=0x7f14040b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13e80039e0 0x7f13e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e8004340 0x7f13e8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8005020 0x7f13e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13e800bf00 con 0x7f13e8004340 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13e800c0b0 con 0x7f13e8005020 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f13e800c2b0 con 0x7f13e80039e0 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140406ce80 0x7f14040add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.440+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e8004340 0x7f13e8004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f1403400640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8005020 0x7f13e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f1402a00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13e80039e0 0x7f13e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f13fbe00640 1 -- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13e80039e0 msgr2=0x7f13e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13e80039e0 0x7f13e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f13fbe00640 1 -- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8005020 msgr2=0x7f13e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8005020 0x7f13e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f13fbe00640 1 -- 172.21.15.106:0/4292409664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e8003520 con 0x7f13e8004340 2024-04-26T21:47:52.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.441+0000 7f1402a00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13e80039e0 0x7f13e8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.442+0000 7f13fbe00640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e8004340 0x7f13e8004820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f14040d0600 tx=0x7f13e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215848ebd0d63c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.442+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec036070 con 0x7f13e8004340 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.442+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ec004030 con 0x7f13e8004340 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.442+0000 7f1402000640 1 -- 172.21.15.106:0/4292409664 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0097d0 con 0x7f13e8004340 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 -- 172.21.15.106:0/4292409664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e8004340 msgr2=0x7f13e8004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 --2- 172.21.15.106:0/4292409664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e8004340 0x7f13e8004820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f14040d0600 tx=0x7f13e8005830 comp rx=0 tx=0).stop 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1403400640 1 -- 172.21.15.106:0/4292409664 reap_dead start 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 -- 172.21.15.106:0/4292409664 shutdown_connections 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 -- 172.21.15.106:0/4292409664 >> 172.21.15.106:0/4292409664 conn(0x7f14040b2d50 msgr2=0x7f14040b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 -- 172.21.15.106:0/4292409664 shutdown_connections 2024-04-26T21:47:52.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.443+0000 7f1409800640 1 -- 172.21.15.106:0/4292409664 wait complete. 2024-04-26T21:47:52.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.445+0000 7f1409800640 1 Processor -- start 2024-04-26T21:47:52.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.445+0000 7f1409800640 1 -- start start 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8004340 0x7f14040b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14040ca170 0x7f14040ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f140415d1d0 con 0x7f140414a070 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14040b05b0 con 0x7f13e8004340 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1409800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f140415d350 con 0x7f14040ca170 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39666/0 (socket says 172.21.15.106:39666) 2024-04-26T21:47:52.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 -- 172.21.15.106:0/198707644 learned_addr learned my addr 172.21.15.106:0/198707644 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:52.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f1402a00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8004340 0x7f14040b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 -- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14040ca170 msgr2=0x7f14040ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14040ca170 0x7f14040ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 -- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8004340 msgr2=0x7f14040b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8004340 0x7f14040b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.446+0000 7f13fbe00640 1 -- 172.21.15.106:0/198707644 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041f4780 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f13fbe00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f13ec02abe0 tx=0x7f13ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec040020 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ec003c40 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041f4a40 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec036040 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.447+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14041f5000 con 0x7f140414a070 2024-04-26T21:47:52.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.448+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1404069540 con 0x7f140414a070 2024-04-26T21:47:52.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.449+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f13ec043020 con 0x7f140414a070 2024-04-26T21:47:52.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.450+0000 7f13faa00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 0x7f13e006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.450+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f13ec0ac400 con 0x7f140414a070 2024-04-26T21:47:52.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.451+0000 7f1402a00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 0x7f13e006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.451+0000 7f1402a00640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 0x7f13e006ef90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.455+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13ec04a020 con 0x7f140414a070 2024-04-26T21:47:52.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.663+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1404069c70 con 0x7f140414a070 2024-04-26T21:47:52.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.665+0000 7f13faa00640 1 -- 172.21.15.106:0/198707644 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13032 (secure 0 0 0) 0x7f13ec07a810 con 0x7f140414a070 2024-04-26T21:47:52.666 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:52.667 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:51.208507+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/107496404":"2024-04-27T21:47:48.197557+0000","172.21.15.204:6833/107496404":"2024-04-27T21:47:48.197557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:52.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 msgr2=0x7f13e006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 0x7f13e006ef90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4009040 comp rx=0 tx=0).stop 2024-04-26T21:47:52.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 msgr2=0x7f14040b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f13ec02abe0 tx=0x7f13ec004730 comp rx=0 tx=0).stop 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 shutdown_connections 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f13e006cad0 0x7f13e006ef90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14040ca170 0x7f14040ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f140414a070 0x7f14040b2190 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 --2- 172.21.15.106:0/198707644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13e8004340 0x7f14040b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.668+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 >> 172.21.15.106:0/198707644 conn(0x7f14040b2d50 msgr2=0x7f14040b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.669+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 shutdown_connections 2024-04-26T21:47:52.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.669+0000 7f1409800640 1 -- 172.21.15.106:0/198707644 wait complete. 2024-04-26T21:47:52.688 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:52.688 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:52.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.888+0000 7f99f7200640 1 Processor -- start 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.889+0000 7f99f7200640 1 -- start start 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.890+0000 7f99f7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99f80d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.890+0000 7f99f7200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80d0170 con 0x7f99f80d2c50 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.890+0000 7f99f7200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80d0450 con 0x7f99f815f7f0 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.890+0000 7f99f7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80d0730 con 0x7f99f80cfb90 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.890+0000 7f99f6800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f99f80d2c50 0x7f99f80d30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54748/0 (socket says 172.21.15.106:54748) 2024-04-26T21:47:52.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f6800640 1 -- 172.21.15.106:0/2764467583 learned_addr learned my addr 172.21.15.106:0/2764467583 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:52.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f5400640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99f80d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053213196 0 0) 0x7f99f80d0170 con 0x7f99f80d2c50 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99dc003520 con 0x7f99f80d2c50 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65136173 0 0) 0x7f99f80d0450 con 0x7f99f815f7f0 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99f80d0170 con 0x7f99f815f7f0 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.891+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3616596234 0 0) 0x7f99dc003520 con 0x7f99f80d2c50 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 msgr2=0x7f99f80d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99f80d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 >> v1:172.21.15.204:6789/0 conn(0x7f99f815f7f0 legacy=0x7f99f815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80d0a10 con 0x7f99f80d2c50 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f99f00015f0 con 0x7f99f80d2c50 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 >> v1:172.21.15.106:6789/0 conn(0x7f99f80d2c50 legacy=0x7f99f80d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99dc003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99dc004190 0x7f99dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99dc004d40 0x7f99dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80d0730 con 0x7f99dc004d40 2024-04-26T21:47:52.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00bc00 con 0x7f99dc004190 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc00be00 con 0x7f99f80cfb90 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f6800640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99dc004d40 0x7f99dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f5e00640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99dc003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.892+0000 7f99f5400640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99dc004190 0x7f99dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.893+0000 7f99f6800640 1 -- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 msgr2=0x7f99dc003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.893+0000 7f99f6800640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99f80cfb90 0x7f99dc003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.893+0000 7f99f6800640 1 -- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99dc004190 msgr2=0x7f99dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.893+0000 7f99f6800640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99dc004190 0x7f99dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.893+0000 7f99f6800640 1 -- 172.21.15.106:0/2764467583 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc003520 con 0x7f99dc004d40 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f6800640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99dc004d40 0x7f99dc0051c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f99f0007f60 tx=0x7f99dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d035189b5b8997 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007b80 con 0x7f99dc004d40 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0005bb0 con 0x7f99dc004d40 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f4a00640 1 -- 172.21.15.106:0/2764467583 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007070 con 0x7f99dc004d40 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f7200640 1 -- 172.21.15.106:0/2764467583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99dc004d40 msgr2=0x7f99dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f7200640 1 --2- 172.21.15.106:0/2764467583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99dc004d40 0x7f99dc0051c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f99f0007f60 tx=0x7f99dc005830 comp rx=0 tx=0).stop 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.894+0000 7f99f6800640 1 -- 172.21.15.106:0/2764467583 reap_dead start 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.895+0000 7f99f7200640 1 -- 172.21.15.106:0/2764467583 shutdown_connections 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.895+0000 7f99f7200640 1 -- 172.21.15.106:0/2764467583 >> 172.21.15.106:0/2764467583 conn(0x7f99f80d4f80 msgr2=0x7f99f80d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.895+0000 7f99f7200640 1 -- 172.21.15.106:0/2764467583 shutdown_connections 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.895+0000 7f99f7200640 1 -- 172.21.15.106:0/2764467583 wait complete. 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.896+0000 7f99f7200640 1 Processor -- start 2024-04-26T21:47:52.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.897+0000 7f99f7200640 1 -- start start 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99dc004d40 0x7f99f8150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99f80cc130 0x7f99f80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99f815f5b0 con 0x7f99f80cddc0 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99f815f430 con 0x7f99f80cc130 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80d22d0 con 0x7f99dc004d40 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39696/0 (socket says 172.21.15.106:39696) 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 -- 172.21.15.106:0/1418959711 learned_addr learned my addr 172.21.15.106:0/1418959711 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5e00640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99dc004d40 0x7f99f8150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 -- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99dc004d40 msgr2=0x7f99f8150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99dc004d40 0x7f99f8150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 -- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99f80cc130 msgr2=0x7f99f80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99f80cc130 0x7f99f80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.898+0000 7f99f5400640 1 -- 172.21.15.106:0/1418959711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80cce60 con 0x7f99f80cddc0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99f5400640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007f90 tx=0x7f99e0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e001a020 con 0x7f99f80cddc0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e0016470 con 0x7f99f80cddc0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0016840 con 0x7f99f80cddc0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f80cd140 con 0x7f99f80cddc0 2024-04-26T21:47:52.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.899+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f8150880 con 0x7f99f80cddc0 2024-04-26T21:47:52.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.900+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f80693c0 con 0x7f99f80cddc0 2024-04-26T21:47:52.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.901+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f99e0005d10 con 0x7f99f80cddc0 2024-04-26T21:47:52.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.902+0000 7f99ee000640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 0x7f99d406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.902+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f99e008cc10 con 0x7f99f80cddc0 2024-04-26T21:47:52.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.902+0000 7f99f5e00640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 0x7f99d406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.903+0000 7f99f5e00640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 0x7f99d406ef90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007850 tx=0x7f99e80077e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:52.906+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99e0024030 con 0x7f99f80cddc0 2024-04-26T21:47:53.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.107+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99f808ea90 con 0x7f99f80cddc0 2024-04-26T21:47:53.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.108+0000 7f99ee000640 1 -- 172.21.15.106:0/1418959711 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f99e005af10 con 0x7f99f80cddc0 2024-04-26T21:47:53.109 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:53.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 msgr2=0x7f99d406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 0x7f99d406ef90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007850 tx=0x7f99e80077e0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 msgr2=0x7f99f80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007f90 tx=0x7f99e0011660 comp rx=0 tx=0).stop 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 shutdown_connections 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99f80cc130 0x7f99f80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99f80cddc0 0x7f99f80ce220 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f99d406cad0 0x7f99d406ef90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 --2- 172.21.15.106:0/1418959711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99dc004d40 0x7f99f8150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.111+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 >> 172.21.15.106:0/1418959711 conn(0x7f99f80d4f80 msgr2=0x7f99f8161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.112+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 shutdown_connections 2024-04-26T21:47:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.112+0000 7f99f7200640 1 -- 172.21.15.106:0/1418959711 wait complete. 2024-04-26T21:47:53.133 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-26T21:47:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.332+0000 7ff647600640 1 Processor -- start 2024-04-26T21:47:53.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.332+0000 7ff647600640 1 -- start start 2024-04-26T21:47:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.334+0000 7ff647600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 0x7ff6401638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.334+0000 7ff647600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff64015f740 0x7ff64015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.334+0000 7ff647600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6400b5fb0 con 0x7ff6401598a0 2024-04-26T21:47:53.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.334+0000 7ff647600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff640159c80 con 0x7ff64015f740 2024-04-26T21:47:53.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.334+0000 7ff647600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff6401600b0 con 0x7ff64006ce80 2024-04-26T21:47:53.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff64015f740 0x7ff64015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff64015f740 0x7ff64015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39708/0 (socket says 172.21.15.106:39708) 2024-04-26T21:47:53.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 -- 172.21.15.106:0/4219319183 learned_addr learned my addr 172.21.15.106:0/4219319183 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff637e00640 1 --2- 172.21.15.106:0/4219319183 >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 0x7ff6401638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 -- 172.21.15.106:0/4219319183 >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 msgr2=0x7ff6401638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 --2- 172.21.15.106:0/4219319183 >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 0x7ff6401638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 -- 172.21.15.106:0/4219319183 >> v1:172.21.15.204:6789/0 conn(0x7ff6401598a0 legacy=0x7ff64006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff645600640 1 -- 172.21.15.106:0/4219319183 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff640160370 con 0x7ff64015f740 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.335+0000 7ff637e00640 1 --2- 172.21.15.106:0/4219319183 >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 0x7ff6401638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.337+0000 7ff645600640 1 --2- 172.21.15.106:0/4219319183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64015f740 0x7ff64015fb70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff6400b3900 tx=0x7ff63802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25e7519253d0c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.337+0000 7ff63fe00640 1 -- 172.21.15.106:0/4219319183 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638036070 con 0x7ff64015f740 2024-04-26T21:47:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.337+0000 7ff63fe00640 1 -- 172.21.15.106:0/4219319183 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff638031600 con 0x7ff64015f740 2024-04-26T21:47:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.337+0000 7ff63fe00640 1 -- 172.21.15.106:0/4219319183 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6380319d0 con 0x7ff64015f740 2024-04-26T21:47:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 -- 172.21.15.106:0/4219319183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64015f740 msgr2=0x7ff64015fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 --2- 172.21.15.106:0/4219319183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64015f740 0x7ff64015fb70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff6400b3900 tx=0x7ff63802a660 comp rx=0 tx=0).stop 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 -- 172.21.15.106:0/4219319183 shutdown_connections 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 --2- 172.21.15.106:0/4219319183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64015f740 0x7ff64015fb70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 --2- 172.21.15.106:0/4219319183 >> v2:172.21.15.204:3300/0 conn(0x7ff64006ce80 0x7ff6401638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 -- 172.21.15.106:0/4219319183 >> 172.21.15.106:0/4219319183 conn(0x7ff6400b2d50 msgr2=0x7ff6400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 -- 172.21.15.106:0/4219319183 shutdown_connections 2024-04-26T21:47:53.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.338+0000 7ff647600640 1 -- 172.21.15.106:0/4219319183 wait complete. 2024-04-26T21:47:53.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.339+0000 7ff647600640 1 Processor -- start 2024-04-26T21:47:53.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.340+0000 7ff647600640 1 -- start start 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.340+0000 7ff647600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.340+0000 7ff647600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6401598a0 0x7ff6401615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff647600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff64015f740 0x7ff6401f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff647600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff640163390 con 0x7ff64006ce80 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff647600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401630b0 con 0x7ff6401598a0 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff647600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff640160650 con 0x7ff64015f740 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39714/0 (socket says 172.21.15.106:39714) 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 -- 172.21.15.106:0/1437862258 learned_addr learned my addr 172.21.15.106:0/1437862258 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 -- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff64015f740 msgr2=0x7ff6401f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff64015f740 0x7ff6401f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 -- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6401598a0 msgr2=0x7ff6401615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6401598a0 0x7ff6401615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 -- 172.21.15.106:0/1437862258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6401f4b50 con 0x7ff64006ce80 2024-04-26T21:47:53.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.341+0000 7ff644c00640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c430 tx=0x7ff63000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.342+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630011790 con 0x7ff64006ce80 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.342+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff630017070 con 0x7ff64006ce80 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.342+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630011bb0 con 0x7ff64006ce80 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.342+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6401f4dc0 con 0x7ff64006ce80 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.342+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6401f8e00 con 0x7ff64006ce80 2024-04-26T21:47:53.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.343+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff640069540 con 0x7ff64006ce80 2024-04-26T21:47:53.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.344+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff63001c460 con 0x7ff64006ce80 2024-04-26T21:47:53.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.345+0000 7ff63d600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 0x7ff61c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.346+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff63008bb30 con 0x7ff64006ce80 2024-04-26T21:47:53.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.346+0000 7ff637e00640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 0x7ff61c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.346+0000 7ff637e00640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 0x7ff61c06ef60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff628006fd0 tx=0x7ff628008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.350+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff630059f40 con 0x7ff64006ce80 2024-04-26T21:47:53.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:53.600+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7ff640073fc0 con 0x7ff64006ce80 2024-04-26T21:47:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.230+0000 7ff63d600640 1 -- 172.21.15.106:0/1437862258 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v129) ==== 143+0+0 (secure 0 0 0) 0x7ff630059840 con 0x7ff64006ce80 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 msgr2=0x7ff61c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 0x7ff61c06ef60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff628006fd0 tx=0x7ff628008040 comp rx=0 tx=0).stop 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 msgr2=0x7ff6401610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c430 tx=0x7ff63000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 shutdown_connections 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff61c06caa0 0x7ff61c06ef60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff64015f740 0x7ff6401f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6401598a0 0x7ff6401615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 --2- 172.21.15.106:0/1437862258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff64006ce80 0x7ff6401610a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 >> 172.21.15.106:0/1437862258 conn(0x7ff6400b2d50 msgr2=0x7ff64006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 shutdown_connections 2024-04-26T21:47:54.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.234+0000 7ff647600640 1 -- 172.21.15.106:0/1437862258 wait complete. 2024-04-26T21:47:54.255 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:47:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.448+0000 7fefc4a00640 1 Processor -- start 2024-04-26T21:47:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.448+0000 7fefc4a00640 1 -- start start 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.449+0000 7fefc4a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fefc0160d60 0x7fefc00b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.449+0000 7fefc4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0146280 0x7fefc01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.449+0000 7fefc4a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc00b4d20 con 0x7fefc0161140 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.449+0000 7fefc4a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fefc006e5f0 con 0x7fefc0160d60 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.449+0000 7fefc4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefc0146c30 con 0x7fefc0146280 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbf400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fefc0160d60 0x7fefc00b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefb7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fefc0161140 0x7fefc006dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54760/0 (socket says 172.21.15.106:54760) 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefb7e00640 1 -- 172.21.15.106:0/1016162365 learned_addr learned my addr 172.21.15.106:0/1016162365 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbfe00640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0146280 0x7fefc01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbf400640 1 -- 172.21.15.106:0/1016162365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0146280 msgr2=0x7fefc01466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbf400640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0146280 0x7fefc01466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbf400640 1 -- 172.21.15.106:0/1016162365 >> v1:172.21.15.106:6789/0 conn(0x7fefc0161140 legacy=0x7fefc006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.450+0000 7fefbf400640 1 -- 172.21.15.106:0/1016162365 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc00b5000 con 0x7fefc0160d60 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.451+0000 7fefbf400640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc00b5b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fefb00097d0 tx=0x7fefb002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e39a3917b6051e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefbea00640 1 -- 172.21.15.106:0/1016162365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0037070 con 0x7fefc0160d60 2024-04-26T21:47:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefbea00640 1 -- 172.21.15.106:0/1016162365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb0032440 con 0x7fefc0160d60 2024-04-26T21:47:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefbea00640 1 -- 172.21.15.106:0/1016162365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0032810 con 0x7fefc0160d60 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 -- 172.21.15.106:0/1016162365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 msgr2=0x7fefc00b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc00b5b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fefb00097d0 tx=0x7fefb002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 -- 172.21.15.106:0/1016162365 shutdown_connections 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0146280 0x7fefc01466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1016162365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc00b5b30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 -- 172.21.15.106:0/1016162365 >> 172.21.15.106:0/1016162365 conn(0x7fefc00b2900 msgr2=0x7fefc00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 -- 172.21.15.106:0/1016162365 shutdown_connections 2024-04-26T21:47:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.452+0000 7fefc4a00640 1 -- 172.21.15.106:0/1016162365 wait complete. 2024-04-26T21:47:54.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.454+0000 7fefc4a00640 1 Processor -- start 2024-04-26T21:47:54.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.454+0000 7fefc4a00640 1 -- start start 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc0146280 0x7fefc0141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0161140 0x7fefc0159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00cf6f0 con 0x7fefc0160d60 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00cf410 con 0x7fefc0146280 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.455+0000 7fefc4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00b6070 con 0x7fefc0161140 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39734/0 (socket says 172.21.15.106:39734) 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 -- 172.21.15.106:0/1448821367 learned_addr learned my addr 172.21.15.106:0/1448821367 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefbf400640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc0146280 0x7fefc0141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefbfe00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0161140 0x7fefc0159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 -- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0161140 msgr2=0x7fefc0159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0161140 0x7fefc0159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 -- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc0146280 msgr2=0x7fefc0141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc0146280 0x7fefc0141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 -- 172.21.15.106:0/1448821367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc0142460 con 0x7fefc0160d60 2024-04-26T21:47:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.456+0000 7fefb7e00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c400 tx=0x7fefa800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.457+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8017070 con 0x7fefc0160d60 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.457+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa8005ce0 con 0x7fefc0160d60 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.457+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc0142740 con 0x7fefc0160d60 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.457+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80072f0 con 0x7fefc0160d60 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.457+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc0142fe0 con 0x7fefc0160d60 2024-04-26T21:47:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.458+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc0142910 con 0x7fefc0160d60 2024-04-26T21:47:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.459+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fefa801a070 con 0x7fefc0160d60 2024-04-26T21:47:54.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.460+0000 7fefb7400640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 0x7fef9c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.460+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fefa808af00 con 0x7fefc0160d60 2024-04-26T21:47:54.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.460+0000 7fefbf400640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 0x7fef9c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.461+0000 7fefbf400640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 0x7fef9c06ef60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefb0034000 tx=0x7fefb002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.464+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefa8059440 con 0x7fefc0160d60 2024-04-26T21:47:54.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.665+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fefc01431e0 con 0x7fefc0160d60 2024-04-26T21:47:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.672+0000 7fefb7400640 1 -- 172.21.15.106:0/1448821367 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fefc01431e0 con 0x7fefc0160d60 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 msgr2=0x7fef9c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 0x7fef9c06ef60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefb0034000 tx=0x7fefb002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 msgr2=0x7fefc01f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c400 tx=0x7fefa800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 shutdown_connections 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fef9c06caa0 0x7fef9c06ef60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc0161140 0x7fefc0159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc0160d60 0x7fefc01f6160 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 --2- 172.21.15.106:0/1448821367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc0146280 0x7fefc0141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 >> 172.21.15.106:0/1448821367 conn(0x7fefc00b2900 msgr2=0x7fefc015d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 shutdown_connections 2024-04-26T21:47:54.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:54.675+0000 7fefc4a00640 1 -- 172.21.15.106:0/1448821367 wait complete. 2024-04-26T21:47:54.695 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:47:54.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:54.796 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:54.909 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:47:54.910 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:54.910 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:54.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:54.968 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:47:54.968 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:55.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:55.026 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:47:55.026 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:55.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:55.082 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:47:55.082 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:55.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:55.139 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:47:55.139 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:55.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.346+0000 7f96f4e00640 1 Processor -- start 2024-04-26T21:47:55.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.346+0000 7f96f4e00640 1 -- start start 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96f4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 0x7f96f006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96f4e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96f015cb70 0x7f96f0073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96f4e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f006d090 con 0x7f96f015cfd0 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96f4e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f96f00af460 con 0x7f96f015cb70 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96f4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96f00ae850 con 0x7f96f006c770 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96ee800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f96f015cfd0 0x7f96f0073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54774/0 (socket says 172.21.15.106:54774) 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96ee800640 1 -- 172.21.15.106:0/3484578611 learned_addr learned my addr 172.21.15.106:0/3484578611 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:55.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.348+0000 7f96e5400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96f015cb70 0x7f96f0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96ede00640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 0x7f96f006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96e5400640 1 -- 172.21.15.106:0/3484578611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 msgr2=0x7f96f006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96e5400640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 0x7f96f006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96e5400640 1 -- 172.21.15.106:0/3484578611 >> v1:172.21.15.106:6789/0 conn(0x7f96f015cfd0 legacy=0x7f96f0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96e5400640 1 -- 172.21.15.106:0/3484578611 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f00aeb30 con 0x7f96f015cb70 2024-04-26T21:47:55.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96ede00640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 0x7f96f006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:55.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.349+0000 7f96ed400640 1 -- 172.21.15.106:0/3484578611 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332906091 0 0) 0x7f96f006d090 con 0x7f96f015cfd0 2024-04-26T21:47:55.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.350+0000 7f96e5400640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cb70 0x7f96f0073120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1b70dffeb30a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.350+0000 7f96ed400640 1 -- 172.21.15.106:0/3484578611 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8037070 con 0x7f96f015cb70 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.350+0000 7f96ed400640 1 -- 172.21.15.106:0/3484578611 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8032440 con 0x7f96f015cb70 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.350+0000 7f96ed400640 1 -- 172.21.15.106:0/3484578611 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8032810 con 0x7f96f015cb70 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 -- 172.21.15.106:0/3484578611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cb70 msgr2=0x7f96f0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cb70 0x7f96f0073120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a180 comp rx=0 tx=0).stop 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 -- 172.21.15.106:0/3484578611 shutdown_connections 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cb70 0x7f96f0073120 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 --2- 172.21.15.106:0/3484578611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f006c770 0x7f96f006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 -- 172.21.15.106:0/3484578611 >> 172.21.15.106:0/3484578611 conn(0x7f96f00b2d50 msgr2=0x7f96f00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 -- 172.21.15.106:0/3484578611 shutdown_connections 2024-04-26T21:47:55.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.351+0000 7f96f4e00640 1 -- 172.21.15.106:0/3484578611 wait complete. 2024-04-26T21:47:55.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.352+0000 7f96f4e00640 1 Processor -- start 2024-04-26T21:47:55.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.353+0000 7f96f4e00640 1 -- start start 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.353+0000 7f96f4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c770 0x7f96f0147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.353+0000 7f96f4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f015cb70 0x7f96f0072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96f4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96f4e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96f015c600 con 0x7f96f015cfd0 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96f4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00af0f0 con 0x7f96f006c770 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96f4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96f015c780 con 0x7f96f015cb70 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39742/0 (socket says 172.21.15.106:39742) 2024-04-26T21:47:55.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 -- 172.21.15.106:0/1122109892 learned_addr learned my addr 172.21.15.106:0/1122109892 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96e5400640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f015cb70 0x7f96f0072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 -- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f015cb70 msgr2=0x7f96f0072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f015cb70 0x7f96f0072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 -- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c770 msgr2=0x7f96f0147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c770 0x7f96f0147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.354+0000 7f96ee800640 1 -- 172.21.15.106:0/1122109892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f0073390 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96ee800640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c410 tx=0x7f96e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e80074e0 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e80047e0 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004bd0 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f0073610 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.355+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f014a070 con 0x7f96f015cfd0 2024-04-26T21:47:55.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.356+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f0149e90 con 0x7f96f015cfd0 2024-04-26T21:47:55.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.357+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f96e8011070 con 0x7f96f015cfd0 2024-04-26T21:47:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.358+0000 7f96e6a00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 0x7f96cc06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.358+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f96e808b7b0 con 0x7f96f015cfd0 2024-04-26T21:47:55.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.358+0000 7f96ede00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 0x7f96cc06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.359+0000 7f96ede00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 0x7f96cc06ef90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f96f01f8a00 tx=0x7f96e00083d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.363+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96f0149e90 con 0x7f96f015cfd0 2024-04-26T21:47:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.603+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f00b5760 con 0x7f96f015cfd0 2024-04-26T21:47:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.605+0000 7f96e6a00640 1 -- 172.21.15.106:0/1122109892 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3880 (secure 0 0 0) 0x7f96f0073390 con 0x7f96f015cfd0 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2307816157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2307816157},{"type":"v1","addr":"172.21.15.106:6836","nonce":2307816157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3707842350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3707842350},{"type":"v1","addr":"172.21.15.106:6837","nonce":3707842350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":34538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/782190866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":782190866},{"type":"v1","addr":"172.21.15.204:6834","nonce":782190866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2203117776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2203117776},{"type":"v1","addr":"172.21.15.204:6835","nonce":2203117776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 19 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.607+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 msgr2=0x7f96cc06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.607+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 0x7f96cc06ef90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f96f01f8a00 tx=0x7f96e00083d0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.607+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 msgr2=0x7f96f014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.607+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c410 tx=0x7f96e800c8d0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.607+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 shutdown_connections 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f015cfd0 0x7f96f014ce50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f96cc06cad0 0x7f96cc06ef90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f015cb70 0x7f96f0072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 --2- 172.21.15.106:0/1122109892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c770 0x7f96f0147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 >> 172.21.15.106:0/1122109892 conn(0x7f96f00b2d50 msgr2=0x7f96f006f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 shutdown_connections 2024-04-26T21:47:55.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.608+0000 7f96f4e00640 1 -- 172.21.15.106:0/1122109892 wait complete. 2024-04-26T21:47:55.628 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:47:55.628 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:47:55.628 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:47:55.628 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:55.629 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:47:55.629 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:47:55.629 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:47:55.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:55.629 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:47:55.629 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:47:55.630 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:47:55.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:55.630 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:47:55.630 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:47:55.630 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:47:55.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:55.646 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:47:55.646 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:47:55.649 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:47:55.649 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:47:55.651 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:47:55.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:47:55.654 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:47:55.654 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:47:55.693 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:47:55.694 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:55.698 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:47:55.699 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:55.700 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:47:55.700 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:55.702 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:47:55.702 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:55.703 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:47:55.838 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-26T21:47:55.842 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-26T21:47:55.851 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-04-26T21:47:55.852 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-26T21:47:55.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.876+0000 7fa720800640 1 Processor -- start 2024-04-26T21:47:55.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.876+0000 7fa720800640 1 -- start start 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa720800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7181598a0 0x7fa7181638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa720800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa71806c770 0x7fa71806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa720800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7180b5fb0 con 0x7fa71815f740 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa720800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa718159c80 con 0x7fa7181598a0 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa720800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa71806d110 con 0x7fa71806c770 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa71de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7181598a0 0x7fa7181638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa71de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7181598a0 0x7fa7181638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39768/0 (socket says 172.21.15.106:39768) 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.877+0000 7fa71de00640 1 -- 172.21.15.106:0/2013440296 learned_addr learned my addr 172.21.15.106:0/2013440296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71d400640 1 --2- 172.21.15.106:0/2013440296 >> v2:172.21.15.204:3300/0 conn(0x7fa71806c770 0x7fa71806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71de00640 1 -- 172.21.15.106:0/2013440296 >> v2:172.21.15.204:3300/0 conn(0x7fa71806c770 msgr2=0x7fa71806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71de00640 1 --2- 172.21.15.106:0/2013440296 >> v2:172.21.15.204:3300/0 conn(0x7fa71806c770 0x7fa71806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71de00640 1 -- 172.21.15.106:0/2013440296 >> v1:172.21.15.106:6789/0 conn(0x7fa71815f740 legacy=0x7fa71815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71de00640 1 -- 172.21.15.106:0/2013440296 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa718160370 con 0x7fa7181598a0 2024-04-26T21:47:55.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.878+0000 7fa71ca00640 1 -- 172.21.15.106:0/2013440296 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658818297 0 0) 0x7fa7180b5fb0 con 0x7fa71815f740 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.879+0000 7fa71de00640 1 --2- 172.21.15.106:0/2013440296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa7181638b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa7180b3900 tx=0x7fa70c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb81e13249f28d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.879+0000 7fa71ca00640 1 -- 172.21.15.106:0/2013440296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c037070 con 0x7fa7181598a0 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.879+0000 7fa71ca00640 1 -- 172.21.15.106:0/2013440296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c032440 con 0x7fa7181598a0 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.879+0000 7fa71ca00640 1 -- 172.21.15.106:0/2013440296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c032810 con 0x7fa7181598a0 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 -- 172.21.15.106:0/2013440296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 msgr2=0x7fa7181638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 --2- 172.21.15.106:0/2013440296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa7181638b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa7180b3900 tx=0x7fa70c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 -- 172.21.15.106:0/2013440296 shutdown_connections 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 --2- 172.21.15.106:0/2013440296 >> v2:172.21.15.204:3300/0 conn(0x7fa71806c770 0x7fa71806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 --2- 172.21.15.106:0/2013440296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa7181638b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 -- 172.21.15.106:0/2013440296 >> 172.21.15.106:0/2013440296 conn(0x7fa7180b2d50 msgr2=0x7fa7180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 -- 172.21.15.106:0/2013440296 shutdown_connections 2024-04-26T21:47:55.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.880+0000 7fa720800640 1 -- 172.21.15.106:0/2013440296 wait complete. 2024-04-26T21:47:55.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.882+0000 7fa720800640 1 Processor -- start 2024-04-26T21:47:55.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.882+0000 7fa720800640 1 -- start start 2024-04-26T21:47:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 0x7fa7180d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 0x7fa7180cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa718163390 con 0x7fa7181598a0 2024-04-26T21:47:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7181630b0 con 0x7fa71815f740 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa720800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa718160650 con 0x7fa71806c770 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39774/0 (socket says 172.21.15.106:39774) 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 -- 172.21.15.106:0/98593426 learned_addr learned my addr 172.21.15.106:0/98593426 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71de00640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 0x7fa7180d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 -- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 msgr2=0x7fa7180d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71e800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 0x7fa7180cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 0x7fa7180d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 -- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 msgr2=0x7fa7180cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 0x7fa7180cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71d400640 1 -- 172.21.15.106:0/98593426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71806f960 con 0x7fa7181598a0 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71de00640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 0x7fa7180d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:55.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.883+0000 7fa71e800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 0x7fa7180cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa71d400640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa70800e8a0 tx=0x7fa70800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708017070 con 0x7fa7181598a0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70800c940 con 0x7fa7181598a0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70800cd30 con 0x7fa7181598a0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71806e380 con 0x7fa7181598a0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.884+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71806e940 con 0x7fa7181598a0 2024-04-26T21:47:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.885+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718069540 con 0x7fa7181598a0 2024-04-26T21:47:55.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.886+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa70801a070 con 0x7fa7181598a0 2024-04-26T21:47:55.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.887+0000 7fa712000640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 0x7fa6fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.887+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa70808be60 con 0x7fa7181598a0 2024-04-26T21:47:55.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.888+0000 7fa71de00640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 0x7fa6fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.888+0000 7fa71de00640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 0x7fa6fc06ef60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:55.892+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa70805a270 con 0x7fa7181598a0 2024-04-26T21:47:56.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:56.098+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa7180cd440 con 0x7fa7181598a0 2024-04-26T21:47:57.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.275+0000 7fa712000640 1 -- 172.21.15.106:0/98593426 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7fa708059b70 con 0x7fa7181598a0 2024-04-26T21:47:57.276 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-26T21:47:57.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 msgr2=0x7fa6fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 0x7fa6fc06ef60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:57.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 msgr2=0x7fa71806f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa70800e8a0 tx=0x7fa70800ed60 comp rx=0 tx=0).stop 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 shutdown_connections 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa71815f740 0x7fa7180cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7181598a0 0x7fa71806f420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa6fc06caa0 0x7fa6fc06ef60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 --2- 172.21.15.106:0/98593426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa71806c770 0x7fa7180d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 >> 172.21.15.106:0/98593426 conn(0x7fa7180b2d50 msgr2=0x7fa7180b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 shutdown_connections 2024-04-26T21:47:57.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.280+0000 7fa720800640 1 -- 172.21.15.106:0/98593426 wait complete. 2024-04-26T21:47:57.302 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:47:57.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.507+0000 7facbdc00640 1 Processor -- start 2024-04-26T21:47:57.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.507+0000 7facbdc00640 1 -- start start 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facbdc00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facb806fa90 0x7facb815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facbdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facbdc00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb806fef0 con 0x7facb8073720 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facbdc00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7facb80701a0 con 0x7facb806fa90 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facbdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7facb806df80 con 0x7facb806ed40 2024-04-26T21:47:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facafe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facb806fa90 0x7facb815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:57.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facafe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facb806fa90 0x7facb815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39786/0 (socket says 172.21.15.106:39786) 2024-04-26T21:47:57.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facb7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7facb8073720 0x7facb8073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54794/0 (socket says 172.21.15.106:54794) 2024-04-26T21:47:57.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.509+0000 7facafe00640 1 -- 172.21.15.106:0/2947631778 learned_addr learned my addr 172.21.15.106:0/2947631778 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:57.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facbd200640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facafe00640 1 -- 172.21.15.106:0/2947631778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 msgr2=0x7facb80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facafe00640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facafe00640 1 -- 172.21.15.106:0/2947631778 >> v1:172.21.15.106:6789/0 conn(0x7facb8073720 legacy=0x7facb8073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facafe00640 1 -- 172.21.15.106:0/2947631778 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb806e260 con 0x7facb806fa90 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facbd200640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.510+0000 7facb7400640 1 -- 172.21.15.106:0/2947631778 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333127532 0 0) 0x7facb806fef0 con 0x7facb8073720 2024-04-26T21:47:57.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.511+0000 7facafe00640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb806fa90 0x7facb815dec0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faca0009870 tx=0x7faca002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34b5ed20578aed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.511+0000 7facb7400640 1 -- 172.21.15.106:0/2947631778 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0037070 con 0x7facb806fa90 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.511+0000 7facb7400640 1 -- 172.21.15.106:0/2947631778 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca0032440 con 0x7facb806fa90 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.511+0000 7facb7400640 1 -- 172.21.15.106:0/2947631778 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0032810 con 0x7facb806fa90 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 -- 172.21.15.106:0/2947631778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb806fa90 msgr2=0x7facb815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb806fa90 0x7facb815dec0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faca0009870 tx=0x7faca002a250 comp rx=0 tx=0).stop 2024-04-26T21:47:57.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 -- 172.21.15.106:0/2947631778 shutdown_connections 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 --2- 172.21.15.106:0/2947631778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb806fa90 0x7facb815dec0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 -- 172.21.15.106:0/2947631778 >> 172.21.15.106:0/2947631778 conn(0x7facb80d0ae0 msgr2=0x7facb80d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 -- 172.21.15.106:0/2947631778 shutdown_connections 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.512+0000 7facbdc00640 1 -- 172.21.15.106:0/2947631778 wait complete. 2024-04-26T21:47:57.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.514+0000 7facbdc00640 1 Processor -- start 2024-04-26T21:47:57.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.514+0000 7facbdc00640 1 -- start start 2024-04-26T21:47:57.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb806fa90 0x7facb80d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7facb8146910 con 0x7facb8073720 2024-04-26T21:47:57.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7facb806f1f0 con 0x7facb806fa90 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7facb8146a90 con 0x7facb806ed40 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39796/0 (socket says 172.21.15.106:39796) 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 -- 172.21.15.106:0/305611687 learned_addr learned my addr 172.21.15.106:0/305611687 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facb7e00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 -- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 msgr2=0x7facb80b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 -- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb806fa90 msgr2=0x7facb80d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb806fa90 0x7facb80d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.515+0000 7facbd200640 1 -- 172.21.15.106:0/305611687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb80b3e20 con 0x7facb8073720 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facbd200640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7facb000cd50 tx=0x7facb000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000f680 con 0x7facb8073720 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb000f840 con 0x7facb8073720 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000fc30 con 0x7facb8073720 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb80b4100 con 0x7facb8073720 2024-04-26T21:47:57.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.516+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb80b49a0 con 0x7facb8073720 2024-04-26T21:47:57.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.518+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb80b44f0 con 0x7facb8073720 2024-04-26T21:47:57.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.519+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7facb00213f0 con 0x7facb8073720 2024-04-26T21:47:57.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.519+0000 7facb4c00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 0x7fac9406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:57.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.520+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7facb008cb00 con 0x7facb8073720 2024-04-26T21:47:57.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.520+0000 7facb7e00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 0x7fac9406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:57.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.520+0000 7facb7e00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 0x7fac9406ef60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faca8007fd0 tx=0x7faca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:57.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.524+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7facb0014030 con 0x7facb8073720 2024-04-26T21:47:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.792+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7facb80707f0 con 0x7facb8073720 2024-04-26T21:47:57.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.793+0000 7facb4c00640 1 -- 172.21.15.106:0/305611687 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7facb80707f0 con 0x7facb8073720 2024-04-26T21:47:57.794 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmnY2XFBAAtaQO3LQwin4e9iNrklJH6w==", 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.795 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxm684cMxAAYTf8acqdf2CCeHokqOS0Xg==", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmLYa9IBAAndFvm1tzi44OqYb6ttR3Sw==", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxm90qUBhAA9teVVNbkqfjkCWMBcZK//Q==", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.797 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBVICxmvpP+ARAAApDQrRCTWu21gf/CGyROOg==", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBYICxmMM6lCxAANvEZrGQcAR75+Xy8ktUM5A==", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.798 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBbICxmZPvaFBAA/2/DymuSvYvc1WYgY1jO/Q==", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBeICxmhYIOIBAAa0ISn+YDALtH4Ma087UI6Q==", 2024-04-26T21:47:57.799 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBkICxmTlgVNhAA3FMFqR8eN9aQooYqEN2kxQ==", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.800 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBoICxmoSRBBhAAVLMhNepkCpcdbgJWdcr8FA==", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBrICxm7HdTEBAA5myIxNTNd3FaNjEbMiILEQ==", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.801 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBuICxmrWiyKRAAxRDKvFrBA1YRdRif8ZH/ow==", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q==", 2024-04-26T21:47:57.802 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmjz8uJRAA7zXqZtFgUCBE9tb+7barEw==", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmCucaGxAAbxoXORUIaoRjav5XMYwKZA==", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBRICxmsp5GLhAAEuL2LpnxmDBCfgz5b3AcuQ==", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-26T21:47:57.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBPICxmLtcjFBAADhgvakKCSkF7zmm7uRGekA==", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.805 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmzUzsDBAAIx87eHODdurs4RGojAb/nQ==", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBQICxmyV5pAhAApZSAbzZOWb572E7vzC6aJQ==", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:57.806 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-26T21:47:57.807 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.807 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:57.807 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-26T21:47:57.807 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.795+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 msgr2=0x7fac9406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.795+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 0x7fac9406ef60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faca8007fd0 tx=0x7faca8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 msgr2=0x7facb813d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7facb000cd50 tx=0x7facb000a650 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 shutdown_connections 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facb8073720 0x7facb813d4d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb806fa90 0x7facb80d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fac9406caa0 0x7fac9406ef60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 --2- 172.21.15.106:0/305611687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb806ed40 0x7facb80b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 >> 172.21.15.106:0/305611687 conn(0x7facb80d0ae0 msgr2=0x7facb80b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 shutdown_connections 2024-04-26T21:47:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.796+0000 7facbdc00640 1 -- 172.21.15.106:0/305611687 wait complete. 2024-04-26T21:47:57.814 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.997+0000 7fccf6200640 1 Processor -- start 2024-04-26T21:47:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.997+0000 7fccf6200640 1 -- start start 2024-04-26T21:47:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.998+0000 7fccf6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf006c770 0x7fccf006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.998+0000 7fccf6200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fccf015cfd0 0x7fccf0073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.998+0000 7fccf6200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf006d090 con 0x7fccf015cb70 2024-04-26T21:47:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.998+0000 7fccf6200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fccf00af460 con 0x7fccf015cfd0 2024-04-26T21:47:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.998+0000 7fccf6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccf00ae850 con 0x7fccf006c770 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fcceea00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fccf015cb70 0x7fccf0073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36842/0 (socket says 172.21.15.106:36842) 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fcceea00640 1 -- 172.21.15.106:0/2223608297 learned_addr learned my addr 172.21.15.106:0/2223608297 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccefe00640 1 --2- 172.21.15.106:0/2223608297 >> v2:172.21.15.106:3300/0 conn(0x7fccf015cfd0 0x7fccf0073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccef400640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf006c770 0x7fccf006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccee000640 1 -- 172.21.15.106:0/2223608297 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330955993 0 0) 0x7fccf006d090 con 0x7fccf015cb70 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccee000640 1 -- 172.21.15.106:0/2223608297 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd4005730 con 0x7fccf015cb70 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccefe00640 1 -- 172.21.15.106:0/2223608297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf006c770 msgr2=0x7fccf006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccefe00640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf006c770 0x7fccf006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccefe00640 1 -- 172.21.15.106:0/2223608297 >> v1:172.21.15.106:6789/0 conn(0x7fccf015cb70 legacy=0x7fccf0073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:57.999+0000 7fccefe00640 1 -- 172.21.15.106:0/2223608297 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf00aeb30 con 0x7fccf015cfd0 2024-04-26T21:47:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.000+0000 7fccefe00640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf015cfd0 0x7fccf0073830 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcce80097a0 tx=0x7fcce802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ca33b28eeab5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccee000640 1 -- 172.21.15.106:0/2223608297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8037070 con 0x7fccf015cfd0 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccee000640 1 -- 172.21.15.106:0/2223608297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce8032440 con 0x7fccf015cfd0 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccee000640 1 -- 172.21.15.106:0/2223608297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8032810 con 0x7fccf015cfd0 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 -- 172.21.15.106:0/2223608297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf015cfd0 msgr2=0x7fccf0073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf015cfd0 0x7fccf0073830 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcce80097a0 tx=0x7fcce802a180 comp rx=0 tx=0).stop 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 -- 172.21.15.106:0/2223608297 shutdown_connections 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf015cfd0 0x7fccf0073830 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 --2- 172.21.15.106:0/2223608297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf006c770 0x7fccf006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.001+0000 7fccf6200640 1 -- 172.21.15.106:0/2223608297 >> 172.21.15.106:0/2223608297 conn(0x7fccf00b2d50 msgr2=0x7fccf00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.002+0000 7fccf6200640 1 -- 172.21.15.106:0/2223608297 shutdown_connections 2024-04-26T21:47:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.002+0000 7fccf6200640 1 -- 172.21.15.106:0/2223608297 wait complete. 2024-04-26T21:47:58.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.003+0000 7fccf6200640 1 Processor -- start 2024-04-26T21:47:58.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.003+0000 7fccf6200640 1 -- start start 2024-04-26T21:47:58.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccf015cb70 0x7fccf0072490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf015cfd0 0x7fccf0144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fccf015c600 con 0x7fccf006c770 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00af0f0 con 0x7fccf015cb70 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.004+0000 7fccf6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccf015c780 con 0x7fccf015cfd0 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46442/0 (socket says 172.21.15.106:46442) 2024-04-26T21:47:58.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 -- 172.21.15.106:0/2435634980 learned_addr learned my addr 172.21.15.106:0/2435634980 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 -- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf015cfd0 msgr2=0x7fccf0144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf015cfd0 0x7fccf0144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 -- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccf015cb70 msgr2=0x7fccf0072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccf015cb70 0x7fccf0072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 -- 172.21.15.106:0/2435634980 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0072d90 con 0x7fccf006c770 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.005+0000 7fccef400640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007f20 tx=0x7fcce000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.006+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0014620 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.006+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce0019070 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.006+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0014a00 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.006+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf0073070 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.006+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf0073630 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.007+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0073ef0 con 0x7fccf006c770 2024-04-26T21:47:58.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.008+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcce0014ba0 con 0x7fccf006c770 2024-04-26T21:47:58.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.009+0000 7fcce7400640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 0x7fccd006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.009+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fcce008c4d0 con 0x7fccf006c770 2024-04-26T21:47:58.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.009+0000 7fcceea00640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 0x7fccd006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.010+0000 7fcceea00640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 0x7fccd006ef60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fccdc000f60 tx=0x7fccdc0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.013+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccf0073ef0 con 0x7fccf006c770 2024-04-26T21:47:58.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.216+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccf006ec40 con 0x7fccf006c770 2024-04-26T21:47:58.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.218+0000 7fcce7400640 1 -- 172.21.15.106:0/2435634980 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12903 (secure 0 0 0) 0x7fcce005a9c0 con 0x7fccf006c770 2024-04-26T21:47:58.219 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:58.219 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":41,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:47:57.273190+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 msgr2=0x7fccd006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 0x7fccd006ef60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fccdc000f60 tx=0x7fccdc0078f0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 msgr2=0x7fccf00cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007f20 tx=0x7fcce000e570 comp rx=0 tx=0).stop 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 shutdown_connections 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccd006caa0 0x7fccd006ef60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccf015cfd0 0x7fccf0144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccf015cb70 0x7fccf0072490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 --2- 172.21.15.106:0/2435634980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf006c770 0x7fccf00cf780 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 >> 172.21.15.106:0/2435634980 conn(0x7fccf00b2d50 msgr2=0x7fccf00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 shutdown_connections 2024-04-26T21:47:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.221+0000 7fccf6200640 1 -- 172.21.15.106:0/2435634980 wait complete. 2024-04-26T21:47:58.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:58.241 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:58.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.439+0000 7feeef000640 1 Processor -- start 2024-04-26T21:47:58.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.440+0000 7feeef000640 1 -- start start 2024-04-26T21:47:58.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee81610d0 0x7feee8167af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 0x7feee8163f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feee806c770 0x7feee806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feee81614b0 con 0x7feee806c770 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feee8164590 con 0x7feee8163af0 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeef000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feee8164870 con 0x7feee81610d0 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.441+0000 7feeed000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feee806c770 0x7feee806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feee806c770 0x7feee806cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46456/0 (socket says 172.21.15.106:46456) 2024-04-26T21:47:58.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 -- 172.21.15.106:0/1504124727 learned_addr learned my addr 172.21.15.106:0/1504124727 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feee7400640 1 --2- 172.21.15.106:0/1504124727 >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 0x7feee8163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 -- 172.21.15.106:0/1504124727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee81610d0 msgr2=0x7feee8167af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feee7e00640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee81610d0 0x7feee8167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee81610d0 0x7feee8167af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 -- 172.21.15.106:0/1504124727 >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 msgr2=0x7feee8163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 --2- 172.21.15.106:0/1504124727 >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 0x7feee8163f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feeed000640 1 -- 172.21.15.106:0/1504124727 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee81672f0 con 0x7feee806c770 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.442+0000 7feee7400640 1 --2- 172.21.15.106:0/1504124727 >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 0x7feee8163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:58.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.443+0000 7feeed000640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee806cc00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feee8075d50 tx=0x7feee002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bac065a06fa420d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.443+0000 7feee6a00640 1 -- 172.21.15.106:0/1504124727 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0036070 con 0x7feee806c770 2024-04-26T21:47:58.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feee6a00640 1 -- 172.21.15.106:0/1504124727 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0031600 con 0x7feee806c770 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feee6a00640 1 -- 172.21.15.106:0/1504124727 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00319d0 con 0x7feee806c770 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 -- 172.21.15.106:0/1504124727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 msgr2=0x7feee806cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee806cc00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feee8075d50 tx=0x7feee002a690 comp rx=0 tx=0).stop 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 -- 172.21.15.106:0/1504124727 shutdown_connections 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee806cc00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 --2- 172.21.15.106:0/1504124727 >> v2:172.21.15.204:3300/0 conn(0x7feee8163af0 0x7feee8163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 --2- 172.21.15.106:0/1504124727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee81610d0 0x7feee8167af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 -- 172.21.15.106:0/1504124727 >> 172.21.15.106:0/1504124727 conn(0x7feee80b2d50 msgr2=0x7feee80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 -- 172.21.15.106:0/1504124727 shutdown_connections 2024-04-26T21:47:58.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.444+0000 7feeef000640 1 -- 172.21.15.106:0/1504124727 wait complete. 2024-04-26T21:47:58.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.446+0000 7feeef000640 1 Processor -- start 2024-04-26T21:47:58.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.446+0000 7feeef000640 1 -- start start 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 0x7feee80cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee8163af0 0x7feee80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feee80b6040 con 0x7feee806c770 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feee80b5ec0 con 0x7feee81610d0 2024-04-26T21:47:58.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feeef000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feee80add80 con 0x7feee8163af0 2024-04-26T21:47:58.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feee7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feee7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46462/0 (socket says 172.21.15.106:46462) 2024-04-26T21:47:58.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.447+0000 7feee7e00640 1 -- 172.21.15.106:0/3649724553 learned_addr learned my addr 172.21.15.106:0/3649724553 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:58.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feeed000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee8163af0 0x7feee80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7400640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 0x7feee80cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 -- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee8163af0 msgr2=0x7feee80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee8163af0 0x7feee80cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 -- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 msgr2=0x7feee80cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 0x7feee80cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 -- 172.21.15.106:0/3649724553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee8151920 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7400640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 0x7feee80cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feee7e00640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feed800c7f0 tx=0x7feed800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8015070 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed8004240 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.448+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8007620 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.449+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee8151c00 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.449+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81521c0 con 0x7feee806c770 2024-04-26T21:47:58.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.450+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee8069650 con 0x7feee806c770 2024-04-26T21:47:58.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.451+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7feed80079d0 con 0x7feee806c770 2024-04-26T21:47:58.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.451+0000 7feedfe00640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 0x7feec406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.452+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7feed808c0a0 con 0x7feee806c770 2024-04-26T21:47:58.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.452+0000 7feee7400640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 0x7feec406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.453+0000 7feee7400640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 0x7feec406ef60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.456+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feed805a480 con 0x7feee806c770 2024-04-26T21:47:58.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.658+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feee80d1a00 con 0x7feee806c770 2024-04-26T21:47:58.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.659+0000 7feedfe00640 1 -- 172.21.15.106:0/3649724553 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7feed8059d80 con 0x7feee806c770 2024-04-26T21:47:58.660 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 msgr2=0x7feec406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 0x7feec406ef60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed0008040 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 msgr2=0x7feee814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feed800c7f0 tx=0x7feed800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 shutdown_connections 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7feec406caa0 0x7feec406ef60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feee8163af0 0x7feee80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee81610d0 0x7feee80cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 --2- 172.21.15.106:0/3649724553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feee806c770 0x7feee814efd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.661+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 >> 172.21.15.106:0/3649724553 conn(0x7feee80b2d50 msgr2=0x7feee80b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.662+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 shutdown_connections 2024-04-26T21:47:58.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.662+0000 7feeef000640 1 -- 172.21.15.106:0/3649724553 wait complete. 2024-04-26T21:47:58.682 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:47:58.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.878+0000 7f6d56400640 1 Processor -- start 2024-04-26T21:47:58.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.878+0000 7f6d56400640 1 -- start start 2024-04-26T21:47:58.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.880+0000 7f6d56400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d50147d50 0x7f6d500cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.880+0000 7f6d56400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500cc240 0x7f6d500c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.880+0000 7f6d56400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5006e680 con 0x7f6d500ca370 2024-04-26T21:47:58.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.880+0000 7f6d56400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6d500cc6a0 con 0x7f6d50147d50 2024-04-26T21:47:58.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.880+0000 7f6d56400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6d500ca800 con 0x7f6d500cc240 2024-04-26T21:47:58.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d50147d50 0x7f6d500cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500cc240 0x7f6d500c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d50147d50 0x7f6d500cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:34726/0 (socket says 172.21.15.106:34726) 2024-04-26T21:47:58.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 -- 172.21.15.106:0/4261901022 learned_addr learned my addr 172.21.15.106:0/4261901022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:58.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 -- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500cc240 msgr2=0x7f6d500c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:58.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 --2- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500cc240 0x7f6d500c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 -- 172.21.15.106:0/4261901022 >> v1:172.21.15.204:6789/0 conn(0x7f6d500ca370 legacy=0x7f6d5006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.881+0000 7f6d55000640 1 -- 172.21.15.106:0/4261901022 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d500caa40 con 0x7f6d50147d50 2024-04-26T21:47:58.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.883+0000 7f6d55000640 1 --2- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d50147d50 0x7f6d500cbd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009870 tx=0x7f6d4002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=748a2dbc655a6dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.883+0000 7f6d4f400640 1 -- 172.21.15.106:0/4261901022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40037070 con 0x7f6d50147d50 2024-04-26T21:47:58.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d4f400640 1 -- 172.21.15.106:0/4261901022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d40032440 con 0x7f6d50147d50 2024-04-26T21:47:58.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d4f400640 1 -- 172.21.15.106:0/4261901022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40032810 con 0x7f6d50147d50 2024-04-26T21:47:58.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 -- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d50147d50 msgr2=0x7f6d500cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 --2- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d50147d50 0x7f6d500cbd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009870 tx=0x7f6d4002a270 comp rx=0 tx=0).stop 2024-04-26T21:47:58.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 -- 172.21.15.106:0/4261901022 shutdown_connections 2024-04-26T21:47:58.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 --2- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500cc240 0x7f6d500c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 --2- 172.21.15.106:0/4261901022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d50147d50 0x7f6d500cbd00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 -- 172.21.15.106:0/4261901022 >> 172.21.15.106:0/4261901022 conn(0x7f6d500d0ae0 msgr2=0x7f6d500d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 -- 172.21.15.106:0/4261901022 shutdown_connections 2024-04-26T21:47:58.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.884+0000 7f6d56400640 1 -- 172.21.15.106:0/4261901022 wait complete. 2024-04-26T21:47:58.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.886+0000 7f6d56400640 1 Processor -- start 2024-04-26T21:47:58.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.886+0000 7f6d56400640 1 -- start start 2024-04-26T21:47:58.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500ca370 0x7f6d50158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d500cc240 0x7f6d50145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5006ec00 con 0x7f6d50147d50 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5006e920 con 0x7f6d500cc240 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d56400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6d500ce6b0 con 0x7f6d500ca370 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d55a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d55a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46464/0 (socket says 172.21.15.106:46464) 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.887+0000 7f6d55a00640 1 -- 172.21.15.106:0/3869693047 learned_addr learned my addr 172.21.15.106:0/3869693047 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d4fe00640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d500cc240 0x7f6d50145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55000640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500ca370 0x7f6d50158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 -- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500ca370 msgr2=0x7f6d50158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500ca370 0x7f6d50158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 -- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d500cc240 msgr2=0x7f6d50145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d500cc240 0x7f6d50145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 -- 172.21.15.106:0/3869693047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d50144d70 con 0x7f6d50147d50 2024-04-26T21:47:58.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d55a00640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c6b0 tx=0x7f6d4800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48003b70 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48003d30 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48005310 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.888+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d50145050 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.889+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d50145610 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.889+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d500693c0 con 0x7f6d50147d50 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.891+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6d480054b0 con 0x7f6d50147d50 2024-04-26T21:47:58.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.892+0000 7f6d4cc00640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 0x7f6d3006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.892+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f6d4808ba20 con 0x7f6d50147d50 2024-04-26T21:47:58.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.892+0000 7f6d55000640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 0x7f6d3006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.892+0000 7f6d55000640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 0x7f6d3006ef90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d40034000 tx=0x7f6d4002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:58.896+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d48059e80 con 0x7f6d50147d50 2024-04-26T21:47:59.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.097+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6d50145980 con 0x7f6d50147d50 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.098+0000 7f6d4cc00640 1 -- 172.21.15.106:0/3869693047 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f6d48059780 con 0x7f6d50147d50 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-26T21:47:59.099 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:47:59.100 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.100+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 msgr2=0x7f6d3006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.100+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 0x7f6d3006ef90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d40034000 tx=0x7f6d4002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.100+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 msgr2=0x7f6d50146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.100+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c6b0 tx=0x7f6d4800cb70 comp rx=0 tx=0).stop 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 shutdown_connections 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d50147d50 0x7f6d50146470 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d500cc240 0x7f6d50145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006cad0 0x7f6d3006ef90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 --2- 172.21.15.106:0/3869693047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d500ca370 0x7f6d50158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 >> 172.21.15.106:0/3869693047 conn(0x7f6d500d0ae0 msgr2=0x7f6d50148130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:59.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 shutdown_connections 2024-04-26T21:47:59.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.101+0000 7f6d56400640 1 -- 172.21.15.106:0/3869693047 wait complete. 2024-04-26T21:47:59.120 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:47:59.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:47:59.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.322+0000 7f004e600640 1 Processor -- start 2024-04-26T21:47:59.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.322+0000 7f004e600640 1 -- start start 2024-04-26T21:47:59.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.324+0000 7f004e600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f004815f680 0x7f00480cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:59.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.324+0000 7f004e600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004806e520 con 0x7f00480ce9f0 2024-04-26T21:47:59.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.324+0000 7f004e600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00480d0810 con 0x7f004815fa60 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.324+0000 7f004e600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f00480b45b0 con 0x7f004815f680 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f004815f680 0x7f00480cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f00480ce9f0 0x7f004806de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36848/0 (socket says 172.21.15.106:36848) 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047e00640 1 -- 172.21.15.106:0/2546205412 learned_addr learned my addr 172.21.15.106:0/2546205412 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0046000640 1 -- 172.21.15.106:0/2546205412 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463031526 0 0) 0x7f004806e520 con 0x7f00480ce9f0 2024-04-26T21:47:59.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0046000640 1 -- 172.21.15.106:0/2546205412 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002c003520 con 0x7f00480ce9f0 2024-04-26T21:47:59.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047400640 1 -- 172.21.15.106:0/2546205412 >> v1:172.21.15.204:6789/0 conn(0x7f004815fa60 legacy=0x7f004806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047400640 1 -- 172.21.15.106:0/2546205412 >> v1:172.21.15.106:6789/0 conn(0x7f00480ce9f0 legacy=0x7f004806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.325+0000 7f0047400640 1 -- 172.21.15.106:0/2546205412 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00480d1e10 con 0x7f004815f680 2024-04-26T21:47:59.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.327+0000 7f0047400640 1 --2- 172.21.15.106:0/2546205412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f004815f680 0x7f00480cf780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00380097a0 tx=0x7f003802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0c14d048287d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:59.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.327+0000 7f0046000640 1 -- 172.21.15.106:0/2546205412 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038037070 con 0x7f004815f680 2024-04-26T21:47:59.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.327+0000 7f0046000640 1 -- 172.21.15.106:0/2546205412 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0038032440 con 0x7f004815f680 2024-04-26T21:47:59.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.327+0000 7f0046000640 1 -- 172.21.15.106:0/2546205412 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00380327f0 con 0x7f004815f680 2024-04-26T21:47:59.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 -- 172.21.15.106:0/2546205412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f004815f680 msgr2=0x7f00480cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 --2- 172.21.15.106:0/2546205412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f004815f680 0x7f00480cf780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00380097a0 tx=0x7f003802a230 comp rx=0 tx=0).stop 2024-04-26T21:47:59.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 -- 172.21.15.106:0/2546205412 shutdown_connections 2024-04-26T21:47:59.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 --2- 172.21.15.106:0/2546205412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f004815f680 0x7f00480cf780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 -- 172.21.15.106:0/2546205412 >> 172.21.15.106:0/2546205412 conn(0x7f00480d2f00 msgr2=0x7f00480d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:59.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 -- 172.21.15.106:0/2546205412 shutdown_connections 2024-04-26T21:47:59.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.328+0000 7f004e600640 1 -- 172.21.15.106:0/2546205412 wait complete. 2024-04-26T21:47:59.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.331+0000 7f004e600640 1 Processor -- start 2024-04-26T21:47:59.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.331+0000 7f004e600640 1 -- start start 2024-04-26T21:47:59.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:59.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004815f680 0x7f004815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:59.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f004815fa60 0x7f004815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:59.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00480d23d0 con 0x7f00480ce9f0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00480cd550 con 0x7f004815f680 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f004e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00480b4280 con 0x7f004815fa60 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f0047400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f0047400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46494/0 (socket says 172.21.15.106:46494) 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.332+0000 7f0047400640 1 -- 172.21.15.106:0/1817312626 learned_addr learned my addr 172.21.15.106:0/1817312626 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 -- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f004815fa60 msgr2=0x7f004815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f004815fa60 0x7f004815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 -- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004815f680 msgr2=0x7f004815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004815f680 0x7f004815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 -- 172.21.15.106:0/1817312626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004815d070 con 0x7f00480ce9f0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f0047400640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00380097a0 tx=0x7f00380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038037050 con 0x7f00480ce9f0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0038043070 con 0x7f00480ce9f0 2024-04-26T21:47:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038032810 con 0x7f00480ce9f0 2024-04-26T21:47:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.333+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00481e89f0 con 0x7f00480ce9f0 2024-04-26T21:47:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.334+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004815d5f0 con 0x7f00480ce9f0 2024-04-26T21:47:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.335+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0048069650 con 0x7f00480ce9f0 2024-04-26T21:47:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.336+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f00380329b0 con 0x7f00480ce9f0 2024-04-26T21:47:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.337+0000 7f003f400640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 0x7f002406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.337+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f00380ac6f0 con 0x7f00480ce9f0 2024-04-26T21:47:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.337+0000 7f0046a00640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 0x7f002406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.338+0000 7f0046a00640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 0x7f002406ef60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f00300079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.341+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f003807ab50 con 0x7f00480ce9f0 2024-04-26T21:47:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:47:59.537+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f004815d7c0 con 0x7f00480ce9f0 2024-04-26T21:48:00.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.305+0000 7f003f400640 1 -- 172.21.15.106:0/1817312626 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7f003807a450 con 0x7f00480ce9f0 2024-04-26T21:48:00.307 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-26T21:48:00.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 msgr2=0x7f002406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 0x7f002406ef60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f00300079b0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 msgr2=0x7f004815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00380097a0 tx=0x7f00380046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 shutdown_connections 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f002406caa0 0x7f002406ef60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f004815fa60 0x7f004815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004815f680 0x7f004815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 --2- 172.21.15.106:0/1817312626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00480ce9f0 0x7f004815db10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.309+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 >> 172.21.15.106:0/1817312626 conn(0x7f00480d2f00 msgr2=0x7f0048071880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.310+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 shutdown_connections 2024-04-26T21:48:00.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.310+0000 7f004e600640 1 -- 172.21.15.106:0/1817312626 wait complete. 2024-04-26T21:48:00.338 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:48:00.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.535+0000 7f6bba200640 1 Processor -- start 2024-04-26T21:48:00.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.535+0000 7f6bba200640 1 -- start start 2024-04-26T21:48:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.536+0000 7f6bba200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6bb40d15d0 0x7f6bb40b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.536+0000 7f6bba200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bb40d3b90 0x7f6bb40b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.536+0000 7f6bba200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb40d0a10 con 0x7f6bb4162850 2024-04-26T21:48:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.536+0000 7f6bba200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb40d0cf0 con 0x7f6bb40d3b90 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.536+0000 7f6bba200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb40b4a80 con 0x7f6bb40d15d0 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb8e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6bb4162850 0x7f6bb40d3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36850/0 (socket says 172.21.15.106:36850) 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb8e00640 1 -- 172.21.15.106:0/1677483945 learned_addr learned my addr 172.21.15.106:0/1677483945 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb9800640 1 --2- 172.21.15.106:0/1677483945 >> v2:172.21.15.106:3300/0 conn(0x7f6bb40d3b90 0x7f6bb40b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb3e00640 1 --2- 172.21.15.106:0/1677483945 >> v2:172.21.15.204:3300/0 conn(0x7f6bb40d15d0 0x7f6bb40b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb3400640 1 -- 172.21.15.106:0/1677483945 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2136171240 0 0) 0x7f6bb40d0a10 con 0x7f6bb4162850 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb3400640 1 -- 172.21.15.106:0/1677483945 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b98003520 con 0x7f6bb4162850 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb9800640 1 -- 172.21.15.106:0/1677483945 >> v2:172.21.15.204:3300/0 conn(0x7f6bb40d15d0 msgr2=0x7f6bb40b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb9800640 1 --2- 172.21.15.106:0/1677483945 >> v2:172.21.15.204:3300/0 conn(0x7f6bb40d15d0 0x7f6bb40b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb9800640 1 -- 172.21.15.106:0/1677483945 >> v1:172.21.15.106:6789/0 conn(0x7f6bb4162850 legacy=0x7f6bb40d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.537+0000 7f6bb9800640 1 -- 172.21.15.106:0/1677483945 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb40b4d60 con 0x7f6bb40d3b90 2024-04-26T21:48:00.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.538+0000 7f6bb9800640 1 --2- 172.21.15.106:0/1677483945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40b57c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6bb408d860 tx=0x7f6bac02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b465f618529f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:00.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bb3400640 1 -- 172.21.15.106:0/1677483945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac007740 con 0x7f6bb40d3b90 2024-04-26T21:48:00.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bb3400640 1 -- 172.21.15.106:0/1677483945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac0078e0 con 0x7f6bb40d3b90 2024-04-26T21:48:00.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bb3400640 1 -- 172.21.15.106:0/1677483945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac007c90 con 0x7f6bb40d3b90 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 -- 172.21.15.106:0/1677483945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40d3b90 msgr2=0x7f6bb40b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 --2- 172.21.15.106:0/1677483945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40b57c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6bb408d860 tx=0x7f6bac02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 -- 172.21.15.106:0/1677483945 shutdown_connections 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 --2- 172.21.15.106:0/1677483945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40b57c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 --2- 172.21.15.106:0/1677483945 >> v2:172.21.15.204:3300/0 conn(0x7f6bb40d15d0 0x7f6bb40b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.539+0000 7f6bba200640 1 -- 172.21.15.106:0/1677483945 >> 172.21.15.106:0/1677483945 conn(0x7f6bb40d4f30 msgr2=0x7f6bb40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.540+0000 7f6bba200640 1 -- 172.21.15.106:0/1677483945 shutdown_connections 2024-04-26T21:48:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.540+0000 7f6bba200640 1 -- 172.21.15.106:0/1677483945 wait complete. 2024-04-26T21:48:00.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.541+0000 7f6bba200640 1 Processor -- start 2024-04-26T21:48:00.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.541+0000 7f6bba200640 1 -- start start 2024-04-26T21:48:00.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.542+0000 7f6bba200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb40d15d0 0x7f6bb409ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.542+0000 7f6bba200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.542+0000 7f6bba200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bba200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40d0740 con 0x7f6bb4162850 2024-04-26T21:48:00.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bba200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40d05c0 con 0x7f6bb40d3b90 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bba200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb40d2c80 con 0x7f6bb40d15d0 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46502/0 (socket says 172.21.15.106:46502) 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 -- 172.21.15.106:0/440424136 learned_addr learned my addr 172.21.15.106:0/440424136 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb3e00640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 -- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb40d15d0 msgr2=0x7f6bb409ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb40d15d0 0x7f6bb409ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 -- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40d3b90 msgr2=0x7f6bb40a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40a2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.543+0000 7f6bb9800640 1 -- 172.21.15.106:0/440424136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb409f170 con 0x7f6bb4162850 2024-04-26T21:48:00.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bb9800640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02abe0 tx=0x7f6bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:00.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac040070 con 0x7f6bb4162850 2024-04-26T21:48:00.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac031c60 con 0x7f6bb4162850 2024-04-26T21:48:00.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac009450 con 0x7f6bb4162850 2024-04-26T21:48:00.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb40a2c80 con 0x7f6bb4162850 2024-04-26T21:48:00.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.544+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb414fac0 con 0x7f6bb4162850 2024-04-26T21:48:00.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.545+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb4050760 con 0x7f6bb4162850 2024-04-26T21:48:00.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.547+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6bac039040 con 0x7f6bb4162850 2024-04-26T21:48:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.548+0000 7f6bb0c00640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.549+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f6bac0ad370 con 0x7f6bb4162850 2024-04-26T21:48:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.549+0000 7f6bb8e00640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.550+0000 7f6bb8e00640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000f60 tx=0x7f6ba40078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.554+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6bac038350 con 0x7f6bb4162850 2024-04-26T21:48:00.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:00.753+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6bb40d3260 con 0x7f6bb4162850 2024-04-26T21:48:01.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.323+0000 7f6bb0c00640 1 -- 172.21.15.106:0/440424136 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f6bac07b5d0 con 0x7f6bb4162850 2024-04-26T21:48:01.323 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-26T21:48:01.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 msgr2=0x7f6b9406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000f60 tx=0x7f6ba40078f0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 msgr2=0x7f6bb4150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02abe0 tx=0x7f6bac004a40 comp rx=0 tx=0).stop 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 shutdown_connections 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bb4162850 0x7f6bb4150880 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bb40d3b90 0x7f6bb40a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b9406caa0 0x7f6b9406ef60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 --2- 172.21.15.106:0/440424136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bb40d15d0 0x7f6bb409ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.327+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 >> 172.21.15.106:0/440424136 conn(0x7f6bb40d4f30 msgr2=0x7f6bb40d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.328+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 shutdown_connections 2024-04-26T21:48:01.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.328+0000 7f6bba200640 1 -- 172.21.15.106:0/440424136 wait complete. 2024-04-26T21:48:01.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:48:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.535+0000 7effeea00640 1 Processor -- start 2024-04-26T21:48:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.535+0000 7effeea00640 1 -- start start 2024-04-26T21:48:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.536+0000 7effeea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 0x7effe815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.536+0000 7effeea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.536+0000 7effeea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe806eea0 con 0x7effe806c770 2024-04-26T21:48:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.536+0000 7effeea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7effe80b4c40 con 0x7effe8156650 2024-04-26T21:48:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.536+0000 7effeea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7effe806d080 con 0x7effe806cbf0 2024-04-26T21:48:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.537+0000 7effeca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.537+0000 7effe7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 0x7effe815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.537+0000 7effeca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:37502/0 (socket says 172.21.15.106:37502) 2024-04-26T21:48:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.537+0000 7effeca00640 1 -- 172.21.15.106:0/785631373 learned_addr learned my addr 172.21.15.106:0/785631373 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.538+0000 7effeca00640 1 -- 172.21.15.106:0/785631373 >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 msgr2=0x7effe815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.538+0000 7effeca00640 1 --2- 172.21.15.106:0/785631373 >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 0x7effe815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.538+0000 7effeca00640 1 -- 172.21.15.106:0/785631373 >> v1:172.21.15.204:6789/0 conn(0x7effe806c770 legacy=0x7effe8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.538+0000 7effeca00640 1 -- 172.21.15.106:0/785631373 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe806f140 con 0x7effe806cbf0 2024-04-26T21:48:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.538+0000 7effe7e00640 1 --2- 172.21.15.106:0/785631373 >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 0x7effe815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:48:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.539+0000 7effeca00640 1 --2- 172.21.15.106:0/785631373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7effe0009870 tx=0x7effe002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d28464ae99f0f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.540+0000 7effe7400640 1 -- 172.21.15.106:0/785631373 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0037070 con 0x7effe806cbf0 2024-04-26T21:48:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.540+0000 7effe7400640 1 -- 172.21.15.106:0/785631373 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe0032440 con 0x7effe806cbf0 2024-04-26T21:48:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.540+0000 7effe7400640 1 -- 172.21.15.106:0/785631373 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0032810 con 0x7effe806cbf0 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.540+0000 7effeea00640 1 -- 172.21.15.106:0/785631373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 msgr2=0x7effe8073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.540+0000 7effeea00640 1 --2- 172.21.15.106:0/785631373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7effe0009870 tx=0x7effe002a250 comp rx=0 tx=0).stop 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 -- 172.21.15.106:0/785631373 shutdown_connections 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 --2- 172.21.15.106:0/785631373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8073830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 --2- 172.21.15.106:0/785631373 >> v2:172.21.15.204:3300/0 conn(0x7effe8156650 0x7effe815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 -- 172.21.15.106:0/785631373 >> 172.21.15.106:0/785631373 conn(0x7effe80b2d50 msgr2=0x7effe80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 -- 172.21.15.106:0/785631373 shutdown_connections 2024-04-26T21:48:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.541+0000 7effeea00640 1 -- 172.21.15.106:0/785631373 wait complete. 2024-04-26T21:48:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.542+0000 7effeea00640 1 Processor -- start 2024-04-26T21:48:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.543+0000 7effeea00640 1 -- start start 2024-04-26T21:48:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effe806c770 0x7effe80734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effe806f9e0 con 0x7effe8156650 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7effe806f700 con 0x7effe806c770 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7effe806fcc0 con 0x7effe806cbf0 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46518/0 (socket says 172.21.15.106:46518) 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 -- 172.21.15.106:0/1931279598 learned_addr learned my addr 172.21.15.106:0/1931279598 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effdf400640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 -- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 msgr2=0x7effe8149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effe7e00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effe806c770 0x7effe80734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 -- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effe806c770 msgr2=0x7effe80734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effe806c770 0x7effe80734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.544+0000 7effeca00640 1 -- 172.21.15.106:0/1931279598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe8149fa0 con 0x7effe8156650 2024-04-26T21:48:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effeca00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7effe0009870 tx=0x7effe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0037040 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe0042070 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe003b930 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe809ba50 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.545+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe815d4a0 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.546+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe8069650 con 0x7effe8156650 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.548+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7effe003bad0 con 0x7effe8156650 2024-04-26T21:48:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.548+0000 7effe4c00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 0x7effc406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.549+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7effe00ac6b0 con 0x7effe8156650 2024-04-26T21:48:01.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.549+0000 7effe7e00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 0x7effc406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:01.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.549+0000 7effe7e00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 0x7effc406ef60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7effe80b5bd0 tx=0x7effd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:01.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.554+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7effe8149fa0 con 0x7effe8156650 2024-04-26T21:48:01.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:01.788+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7effe806ec80 con 0x7effe8156650 2024-04-26T21:48:02.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.343+0000 7effe4c00640 1 -- 172.21.15.106:0/1931279598 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7effe007a820 con 0x7effe8156650 2024-04-26T21:48:02.348 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 6 and data pool 7 2024-04-26T21:48:02.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.354+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 msgr2=0x7effc406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.354+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 0x7effc406ef60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7effe80b5bd0 tx=0x7effd8008040 comp rx=0 tx=0).stop 2024-04-26T21:48:02.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.354+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 msgr2=0x7effe814f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.354+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7effe0009870 tx=0x7effe00046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 shutdown_connections 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effe8156650 0x7effe814f6f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7effc406caa0 0x7effc406ef60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effe806cbf0 0x7effe8149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 --2- 172.21.15.106:0/1931279598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effe806c770 0x7effe80734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.355+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 >> 172.21.15.106:0/1931279598 conn(0x7effe80b2d50 msgr2=0x7effe814ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:02.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.360+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 shutdown_connections 2024-04-26T21:48:02.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.361+0000 7effeea00640 1 -- 172.21.15.106:0/1931279598 wait complete. 2024-04-26T21:48:02.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.555+0000 7f5ebb200640 1 Processor -- start 2024-04-26T21:48:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.556+0000 7f5ebb200640 1 -- start start 2024-04-26T21:48:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ebc145570 0x7f5ebc0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc073ed0 0x7f5ebc0d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ebc15f940 0x7f5ebc15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc1606b0 con 0x7f5ebc145570 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc073110 con 0x7f5ebc15f940 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5ebb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0733f0 con 0x7f5ebc073ed0 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5eb9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ebc145570 0x7f5ebc0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5eb9e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ebc145570 0x7f5ebc0d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46526/0 (socket says 172.21.15.106:46526) 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.557+0000 7f5eb9e00640 1 -- 172.21.15.106:0/540741868 learned_addr learned my addr 172.21.15.106:0/540741868 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.558+0000 7f5eb9e00640 1 -- 172.21.15.106:0/540741868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc073ed0 msgr2=0x7f5ebc0d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.558+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/540741868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc073ed0 0x7f5ebc0d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.558+0000 7f5eb9e00640 1 -- 172.21.15.106:0/540741868 >> v2:172.21.15.204:3300/0 conn(0x7f5ebc15f940 msgr2=0x7f5ebc15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.558+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/540741868 >> v2:172.21.15.204:3300/0 conn(0x7f5ebc15f940 0x7f5ebc15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.558+0000 7f5eb9e00640 1 -- 172.21.15.106:0/540741868 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0736d0 con 0x7f5ebc145570 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/540741868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc145570 0x7f5ebc0d0e70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d0a60 tx=0x7f5eac02c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff7eda91c72e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5eb8a00640 1 -- 172.21.15.106:0/540741868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac039070 con 0x7f5ebc145570 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5eb8a00640 1 -- 172.21.15.106:0/540741868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac034410 con 0x7f5ebc145570 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5eb8a00640 1 -- 172.21.15.106:0/540741868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0347e0 con 0x7f5ebc145570 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5ebb200640 1 -- 172.21.15.106:0/540741868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc145570 msgr2=0x7f5ebc0d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.559+0000 7f5ebb200640 1 --2- 172.21.15.106:0/540741868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc145570 0x7f5ebc0d0e70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d0a60 tx=0x7f5eac02c820 comp rx=0 tx=0).stop 2024-04-26T21:48:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 -- 172.21.15.106:0/540741868 shutdown_connections 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 --2- 172.21.15.106:0/540741868 >> v2:172.21.15.204:3300/0 conn(0x7f5ebc15f940 0x7f5ebc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 --2- 172.21.15.106:0/540741868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc073ed0 0x7f5ebc0d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 --2- 172.21.15.106:0/540741868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc145570 0x7f5ebc0d0e70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 -- 172.21.15.106:0/540741868 >> 172.21.15.106:0/540741868 conn(0x7f5ebc06c740 msgr2=0x7f5ebc06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 -- 172.21.15.106:0/540741868 shutdown_connections 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.560+0000 7f5ebb200640 1 -- 172.21.15.106:0/540741868 wait complete. 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.561+0000 7f5ebb200640 1 Processor -- start 2024-04-26T21:48:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.561+0000 7f5ebb200640 1 -- start start 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc145570 0x7f5ebc0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc15f940 0x7f5ebc0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc1602d0 con 0x7f5ebc073ed0 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc145950 con 0x7f5ebc145570 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5ebb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc160450 con 0x7f5ebc15f940 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.562+0000 7f5eb9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46528/0 (socket says 172.21.15.106:46528) 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 -- 172.21.15.106:0/3375017395 learned_addr learned my addr 172.21.15.106:0/3375017395 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 -- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc15f940 msgr2=0x7f5ebc0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc15f940 0x7f5ebc0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 -- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc145570 msgr2=0x7f5ebc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc145570 0x7f5ebc0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 -- 172.21.15.106:0/3375017395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc15c4c0 con 0x7f5ebc073ed0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb9e00640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5eac037040 tx=0x7f5eac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac044020 con 0x7f5ebc073ed0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.563+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac03f750 con 0x7f5ebc073ed0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.564+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac039040 con 0x7f5ebc073ed0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.564+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc15c7a0 con 0x7f5ebc073ed0 2024-04-26T21:48:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.564+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc15cd60 con 0x7f5ebc073ed0 2024-04-26T21:48:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.565+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0693c0 con 0x7f5ebc073ed0 2024-04-26T21:48:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.566+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5eac047020 con 0x7f5ebc073ed0 2024-04-26T21:48:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.566+0000 7f5eb2000640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 0x7f5e9806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.566+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5eac0ae410 con 0x7f5ebc073ed0 2024-04-26T21:48:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.567+0000 7f5eb9400640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 0x7f5e9806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.567+0000 7f5eb9400640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 0x7f5e9806ef90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0abeb0 tx=0x7f5ea4006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.571+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5eac07c410 con 0x7f5ebc073ed0 2024-04-26T21:48:02.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.758+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ebc0b4850 con 0x7f5ebc073ed0 2024-04-26T21:48:02.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.761+0000 7f5eb2000640 1 -- 172.21.15.106:0/3375017395 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16638 (secure 0 0 0) 0x7f5eac07bd10 con 0x7f5ebc073ed0 2024-04-26T21:48:02.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:02.761 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:01.789885+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 msgr2=0x7f5e9806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 0x7f5e9806ef90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0abeb0 tx=0x7f5ea4006c90 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 msgr2=0x7f5ebc0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5eac037040 tx=0x7f5eac004a40 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 shutdown_connections 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e9806cad0 0x7f5e9806ef90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc15f940 0x7f5ebc0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc145570 0x7f5ebc0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.763+0000 7f5ebb200640 1 --2- 172.21.15.106:0/3375017395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ebc073ed0 0x7f5ebc0ccfa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.764+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 >> 172.21.15.106:0/3375017395 conn(0x7f5ebc06c740 msgr2=0x7f5ebc0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.764+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 shutdown_connections 2024-04-26T21:48:02.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.764+0000 7f5ebb200640 1 -- 172.21.15.106:0/3375017395 wait complete. 2024-04-26T21:48:02.782 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:02.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.980+0000 7fbf89000640 1 Processor -- start 2024-04-26T21:48:02.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.980+0000 7fbf89000640 1 -- start start 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf89000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbf840d0e70 0x7fbf840d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf89000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf840b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf89000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf84146a60 con 0x7fbf84146680 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf89000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8406e970 con 0x7fbf840d0e70 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf89000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf840b60a0 con 0x7fbf840b3b20 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf83400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbf84146680 0x7fbf840b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36852/0 (socket says 172.21.15.106:36852) 2024-04-26T21:48:02.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf83400640 1 -- 172.21.15.106:0/2302563909 learned_addr learned my addr 172.21.15.106:0/2302563909 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99162283 0 0) 0x7fbf84146a60 con 0x7fbf84146680 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf82a00640 1 --2- 172.21.15.106:0/2302563909 >> v2:172.21.15.204:3300/0 conn(0x7fbf840d0e70 0x7fbf840d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf83e00640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf840b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.982+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf68003520 con 0x7fbf84146680 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1334906717 0 0) 0x7fbf68003520 con 0x7fbf84146680 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 msgr2=0x7fbf840b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf840b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 >> v2:172.21.15.204:3300/0 conn(0x7fbf840d0e70 msgr2=0x7fbf840d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 --2- 172.21.15.106:0/2302563909 >> v2:172.21.15.204:3300/0 conn(0x7fbf840d0e70 0x7fbf840d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8406dd60 con 0x7fbf84146680 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fbf74001670 con 0x7fbf84146680 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 >> v1:172.21.15.106:6789/0 conn(0x7fbf84146680 legacy=0x7fbf840b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf680039b0 0x7fbf68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840b60a0 con 0x7fbf680039b0 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800b9c0 con 0x7fbf68004800 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6800bb40 con 0x7fbf840b3b20 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.983+0000 7fbf83400640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf680039b0 0x7fbf68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83e00640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83400640 1 -- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 msgr2=0x7fbf680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83400640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf840b3b20 0x7fbf680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83400640 1 -- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf68004800 msgr2=0x7fbf68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:02.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83400640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.984+0000 7fbf83400640 1 -- 172.21.15.106:0/2302563909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf68003520 con 0x7fbf680039b0 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf83400640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf680039b0 0x7fbf68003dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400c210 tx=0x7fbf68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9dc13eb582569 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74007d50 con 0x7fbf680039b0 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf74005af0 con 0x7fbf680039b0 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf82000640 1 -- 172.21.15.106:0/2302563909 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74007070 con 0x7fbf680039b0 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf89000640 1 -- 172.21.15.106:0/2302563909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf680039b0 msgr2=0x7fbf68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf89000640 1 --2- 172.21.15.106:0/2302563909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf680039b0 0x7fbf68003dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400c210 tx=0x7fbf68005830 comp rx=0 tx=0).stop 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.985+0000 7fbf83e00640 1 -- 172.21.15.106:0/2302563909 reap_dead start 2024-04-26T21:48:02.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.986+0000 7fbf89000640 1 -- 172.21.15.106:0/2302563909 shutdown_connections 2024-04-26T21:48:02.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.986+0000 7fbf89000640 1 -- 172.21.15.106:0/2302563909 >> 172.21.15.106:0/2302563909 conn(0x7fbf8406c740 msgr2=0x7fbf8406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:02.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.986+0000 7fbf89000640 1 -- 172.21.15.106:0/2302563909 shutdown_connections 2024-04-26T21:48:02.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.986+0000 7fbf89000640 1 -- 172.21.15.106:0/2302563909 wait complete. 2024-04-26T21:48:02.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.987+0000 7fbf89000640 1 Processor -- start 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.988+0000 7fbf89000640 1 -- start start 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf680039b0 0x7fbf840b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf84139860 0x7fbf84154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8406e320 con 0x7fbf8415c280 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840d17f0 con 0x7fbf680039b0 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf89000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8406e600 con 0x7fbf84139860 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46552/0 (socket says 172.21.15.106:46552) 2024-04-26T21:48:02.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 -- 172.21.15.106:0/4085492721 learned_addr learned my addr 172.21.15.106:0/4085492721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83400640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf680039b0 0x7fbf840b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf82a00640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf84139860 0x7fbf84154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 -- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf84139860 msgr2=0x7fbf84154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf84139860 0x7fbf84154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 -- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf680039b0 msgr2=0x7fbf840b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf680039b0 0x7fbf840b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.989+0000 7fbf83e00640 1 -- 172.21.15.106:0/4085492721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf84138dc0 con 0x7fbf8415c280 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf83e00640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf8406f440 tx=0x7fbf7c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c01c070 con 0x7fbf8415c280 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c017400 con 0x7fbf8415c280 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf840b47a0 con 0x7fbf8415c280 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0177f0 con 0x7fbf8415c280 2024-04-26T21:48:02.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.990+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf84139340 con 0x7fbf8415c280 2024-04-26T21:48:02.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.991+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf84139540 con 0x7fbf8415c280 2024-04-26T21:48:02.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.992+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fbf7c01f070 con 0x7fbf8415c280 2024-04-26T21:48:02.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.993+0000 7fbf7b400640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 0x7fbf6406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:02.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.993+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c08c190 con 0x7fbf8415c280 2024-04-26T21:48:02.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.993+0000 7fbf83400640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 0x7fbf6406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:02.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.994+0000 7fbf83400640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 0x7fbf6406ef90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf74033000 tx=0x7fbf74009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:02.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:02.998+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf84139540 con 0x7fbf8415c280 2024-04-26T21:48:03.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.185+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf840693c0 con 0x7fbf8415c280 2024-04-26T21:48:03.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.187+0000 7fbf7b400640 1 -- 172.21.15.106:0/4085492721 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16638 (secure 0 0 0) 0x7fbf7c012070 con 0x7fbf8415c280 2024-04-26T21:48:03.188 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:03.188 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:01.789885+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 msgr2=0x7fbf6406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 0x7fbf6406ef90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf74033000 tx=0x7fbf74009030 comp rx=0 tx=0).stop 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 msgr2=0x7fbf84160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf8406f440 tx=0x7fbf7c00f4a0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 shutdown_connections 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf8415c280 0x7fbf84160a20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fbf6406cad0 0x7fbf6406ef90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf84139860 0x7fbf84154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 --2- 172.21.15.106:0/4085492721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf680039b0 0x7fbf840b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 >> 172.21.15.106:0/4085492721 conn(0x7fbf8406c740 msgr2=0x7fbf841413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:03.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 shutdown_connections 2024-04-26T21:48:03.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.190+0000 7fbf89000640 1 -- 172.21.15.106:0/4085492721 wait complete. 2024-04-26T21:48:03.208 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:48:03.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.401+0000 7fdbe1a00640 1 Processor -- start 2024-04-26T21:48:03.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.401+0000 7fdbe1a00640 1 -- start start 2024-04-26T21:48:03.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.402+0000 7fdbe1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc167af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.402+0000 7fdbe1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 0x7fdbdc163f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.402+0000 7fdbe1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc1614b0 con 0x7fdbdc06c770 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.402+0000 7fdbe1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc164590 con 0x7fdbdc163af0 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.402+0000 7fdbe1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc164870 con 0x7fdbdc1610d0 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.403+0000 7fdbda000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 0x7fdbdc163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.403+0000 7fdbda000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 0x7fdbdc163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:34822/0 (socket says 172.21.15.106:34822) 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.403+0000 7fdbda000640 1 -- 172.21.15.106:0/2203249913 learned_addr learned my addr 172.21.15.106:0/2203249913 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.403+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2203249913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.404+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2203249913 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 msgr2=0x7fdbdc163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.404+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2203249913 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 0x7fdbdc163f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.404+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2203249913 >> v1:172.21.15.204:6789/0 conn(0x7fdbdc06c770 legacy=0x7fdbdc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.404+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2203249913 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1672f0 con 0x7fdbdc1610d0 2024-04-26T21:48:03.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.405+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2203249913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc167af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0097a0 tx=0x7fdbcc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314a4d3bc500bd34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:03.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.406+0000 7fdbd9600640 1 -- 172.21.15.106:0/2203249913 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037070 con 0x7fdbdc1610d0 2024-04-26T21:48:03.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.406+0000 7fdbd9600640 1 -- 172.21.15.106:0/2203249913 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc032440 con 0x7fdbdc1610d0 2024-04-26T21:48:03.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.406+0000 7fdbd9600640 1 -- 172.21.15.106:0/2203249913 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc032810 con 0x7fdbdc1610d0 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.406+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2203249913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 msgr2=0x7fdbdc167af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.406+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2203249913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc167af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0097a0 tx=0x7fdbcc02a180 comp rx=0 tx=0).stop 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2203249913 shutdown_connections 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2203249913 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc163af0 0x7fdbdc163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2203249913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc167af0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2203249913 >> 172.21.15.106:0/2203249913 conn(0x7fdbdc0b2d50 msgr2=0x7fdbdc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2203249913 shutdown_connections 2024-04-26T21:48:03.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.407+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2203249913 wait complete. 2024-04-26T21:48:03.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.408+0000 7fdbe1a00640 1 Processor -- start 2024-04-26T21:48:03.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.408+0000 7fdbe1a00640 1 -- start start 2024-04-26T21:48:03.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc163af0 0x7fdbdc09c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b6040 con 0x7fdbdc06c770 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b5ec0 con 0x7fdbdc163af0 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.409+0000 7fdbe1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc06cc00 con 0x7fdbdc1610d0 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46556/0 (socket says 172.21.15.106:46556) 2024-04-26T21:48:03.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2408892453 learned_addr learned my addr 172.21.15.106:0/2408892453 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:03.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 msgr2=0x7fdbdc06ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc163af0 msgr2=0x7fdbdc09c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:03.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc163af0 0x7fdbdc09c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:03.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 -- 172.21.15.106:0/2408892453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1f9150 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbdaa00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098c0 tx=0x7fdbcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.410+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037040 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.411+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc030500 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.411+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc06f6a0 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.411+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0308d0 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.411+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc15e840 con 0x7fdbdc06c770 2024-04-26T21:48:03.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.412+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc06fbc0 con 0x7fdbdc06c770 2024-04-26T21:48:03.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.413+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fdbcc044020 con 0x7fdbdc06c770 2024-04-26T21:48:03.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.414+0000 7fdbd2a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 0x7fdbbc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:03.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.414+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdbcc0ac280 con 0x7fdbdc06c770 2024-04-26T21:48:03.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.414+0000 7fdbda000640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 0x7fdbbc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:03.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.415+0000 7fdbda000640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 0x7fdbbc06ef60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006f40 tx=0x7fdbc8006e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:03.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.419+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdbdc06fbc0 con 0x7fdbdc06c770 2024-04-26T21:48:03.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:03.659+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdbdc069650 con 0x7fdbdc06c770 2024-04-26T21:48:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.355+0000 7fdbd2a00640 1 -- 172.21.15.106:0/2408892453 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7fdbcc07a310 con 0x7fdbdc06c770 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 msgr2=0x7fdbbc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 0x7fdbbc06ef60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006f40 tx=0x7fdbc8006e90 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 msgr2=0x7fdbdc14f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098c0 tx=0x7fdbcc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 shutdown_connections 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc163af0 0x7fdbdc09c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fdbbc06caa0 0x7fdbbc06ef60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc1610d0 0x7fdbdc06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 --2- 172.21.15.106:0/2408892453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc06c770 0x7fdbdc14f260 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.359+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 >> 172.21.15.106:0/2408892453 conn(0x7fdbdc0b2d50 msgr2=0x7fdbdc0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.360+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 shutdown_connections 2024-04-26T21:48:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.360+0000 7fdbe1a00640 1 -- 172.21.15.106:0/2408892453 wait complete. 2024-04-26T21:48:04.380 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:48:04.380 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:48:04.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.586+0000 7f5d12e00640 1 Processor -- start 2024-04-26T21:48:04.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.586+0000 7f5d12e00640 1 -- start start 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.588+0000 7f5d12e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.588+0000 7f5d12e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d0c06c770 0x7f5d0c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.588+0000 7f5d12e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c06eea0 con 0x7f5d0c073120 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.588+0000 7f5d12e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5d0c0b4c40 con 0x7f5d0c06c770 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.588+0000 7f5d12e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c06f140 con 0x7f5d0c156650 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d0c06c770 0x7f5d0c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d0c06c770 0x7f5d0c06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:34838/0 (socket says 172.21.15.106:34838) 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 -- 172.21.15.106:0/663314264 learned_addr learned my addr 172.21.15.106:0/663314264 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 -- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 msgr2=0x7f5d0c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 --2- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 -- 172.21.15.106:0/663314264 >> v1:172.21.15.204:6789/0 conn(0x7f5d0c073120 legacy=0x7f5d0c0735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:04.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.589+0000 7f5d0b400640 1 -- 172.21.15.106:0/663314264 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c06f420 con 0x7f5d0c06c770 2024-04-26T21:48:04.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.591+0000 7f5d0b400640 1 --2- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c06cbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800bb10 tx=0x7f5cf802c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd9964964ce728f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:04.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.591+0000 7f5d0aa00640 1 -- 172.21.15.106:0/663314264 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8032790 con 0x7f5d0c06c770 2024-04-26T21:48:04.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d0aa00640 1 -- 172.21.15.106:0/663314264 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf8038070 con 0x7f5d0c06c770 2024-04-26T21:48:04.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d0aa00640 1 -- 172.21.15.106:0/663314264 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8032b50 con 0x7f5d0c06c770 2024-04-26T21:48:04.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 -- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 msgr2=0x7f5d0c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:04.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 --2- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c06cbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800bb10 tx=0x7f5cf802c8d0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 -- 172.21.15.106:0/663314264 shutdown_connections 2024-04-26T21:48:04.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 --2- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c06cbf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 --2- 172.21.15.106:0/663314264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 -- 172.21.15.106:0/663314264 >> 172.21.15.106:0/663314264 conn(0x7f5d0c0b2d50 msgr2=0x7f5d0c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:04.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 -- 172.21.15.106:0/663314264 shutdown_connections 2024-04-26T21:48:04.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.592+0000 7f5d12e00640 1 -- 172.21.15.106:0/663314264 wait complete. 2024-04-26T21:48:04.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.594+0000 7f5d12e00640 1 Processor -- start 2024-04-26T21:48:04.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.594+0000 7f5d12e00640 1 -- start start 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c06fcc0 con 0x7f5d0c073120 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c06f9e0 con 0x7f5d0c06c770 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d12e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c15d320 con 0x7f5d0c156650 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.595+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46562/0 (socket says 172.21.15.106:46562) 2024-04-26T21:48:04.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 -- 172.21.15.106:0/3506087891 learned_addr learned my addr 172.21.15.106:0/3506087891 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0be00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d10e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 -- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 msgr2=0x7f5d0c1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 -- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 msgr2=0x7f5d0c14f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c14f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:04.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 -- 172.21.15.106:0/3506087891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c1584e0 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.596+0000 7f5d0b400640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b2c0 tx=0x7f5cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.597+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8033620 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.597+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf80337e0 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.597+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c1587c0 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.597+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8033bd0 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.597+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c158d80 con 0x7f5d0c073120 2024-04-26T21:48:04.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.598+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c069650 con 0x7f5d0c073120 2024-04-26T21:48:04.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.599+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5cf804d060 con 0x7f5d0c073120 2024-04-26T21:48:04.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.600+0000 7f5d03e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 0x7f5cec06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:04.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.600+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf80aeb70 con 0x7f5d0c073120 2024-04-26T21:48:04.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.601+0000 7f5d0be00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 0x7f5cec06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:04.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.601+0000 7f5d0be00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 0x7f5cec06ef60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009a30 tx=0x7f5cfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:04.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.605+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5cf8041070 con 0x7f5d0c073120 2024-04-26T21:48:04.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:04.847+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5d0c15e030 con 0x7f5d0c073120 2024-04-26T21:48:05.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.367+0000 7f5d03e00640 1 -- 172.21.15.106:0/3506087891 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f5cf80312e0 con 0x7f5d0c073120 2024-04-26T21:48:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.371+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 msgr2=0x7f5cec06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.371+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 0x7f5cec06ef60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009a30 tx=0x7f5cfc008040 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.371+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 msgr2=0x7f5d0c150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.371+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b2c0 tx=0x7f5cf80046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 shutdown_connections 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5cec06caa0 0x7f5cec06ef60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d0c156650 0x7f5d0c1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d0c073120 0x7f5d0c150180 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 --2- 172.21.15.106:0/3506087891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d0c06c770 0x7f5d0c14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 >> 172.21.15.106:0/3506087891 conn(0x7f5d0c0b2d50 msgr2=0x7f5d0c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 shutdown_connections 2024-04-26T21:48:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.372+0000 7f5d12e00640 1 -- 172.21.15.106:0/3506087891 wait complete. 2024-04-26T21:48:05.394 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:48:05.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.595+0000 7f2059e00640 1 Processor -- start 2024-04-26T21:48:05.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.595+0000 7f2059e00640 1 -- start start 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.596+0000 7f2059e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f205406c770 0x7f205406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2059e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2054073f40 con 0x7f205415cfd0 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2059e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205406d090 con 0x7f205415cb70 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2059e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f20540af460 con 0x7f205406c770 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2053400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f205415cfd0 0x7f2054073830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36858/0 (socket says 172.21.15.106:36858) 2024-04-26T21:48:05.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2053400640 1 -- 172.21.15.106:0/1423874544 learned_addr learned my addr 172.21.15.106:0/1423874544 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2052a00640 1 --2- 172.21.15.106:0/1423874544 >> v2:172.21.15.204:3300/0 conn(0x7f205406c770 0x7f205406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951517093 0 0) 0x7f2054073f40 con 0x7f205415cfd0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.597+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2038003520 con 0x7f205415cfd0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 647687632 0 0) 0x7f2038003520 con 0x7f205415cfd0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 >> v2:172.21.15.204:3300/0 conn(0x7f205406c770 msgr2=0x7f205406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 --2- 172.21.15.106:0/1423874544 >> v2:172.21.15.204:3300/0 conn(0x7f205406c770 0x7f205406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 >> v1:172.21.15.204:6789/0 conn(0x7f205415cb70 legacy=0x7f2054073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20540ae850 con 0x7f205415cfd0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f204c0019f0 con 0x7f205415cfd0 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 >> v1:172.21.15.106:6789/0 conn(0x7f205415cfd0 legacy=0x7f2054073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380039b0 0x7f2038003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f2038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20380051c0 0x7f203800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20540af460 con 0x7f2038004800 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f203800c060 con 0x7f20380039b0 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f203800c260 con 0x7f20380051c0 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.598+0000 7f2052000640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f2038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052a00640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380039b0 0x7f2038003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2053400640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20380051c0 0x7f203800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052000640 1 -- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20380051c0 msgr2=0x7f203800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052000640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20380051c0 0x7f203800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052000640 1 -- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380039b0 msgr2=0x7f2038003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052000640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380039b0 0x7f2038003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2052000640 1 -- 172.21.15.106:0/1423874544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038003520 con 0x7f2038004800 2024-04-26T21:48:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2053400640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20380051c0 0x7f203800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.599+0000 7f2053400640 1 -- 172.21.15.106:0/1423874544 reap_dead start 2024-04-26T21:48:05.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.600+0000 7f2052000640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f2038004c80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2040001750 tx=0x7f2038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26710b53f579ed03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.600+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040001370 con 0x7f2038004800 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.600+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20400052d0 con 0x7f2038004800 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.600+0000 7f2051600640 1 -- 172.21.15.106:0/1423874544 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204002abc0 con 0x7f2038004800 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 -- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 msgr2=0x7f2038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f2038004c80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2040001750 tx=0x7f2038005830 comp rx=0 tx=0).stop 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 -- 172.21.15.106:0/1423874544 shutdown_connections 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 --2- 172.21.15.106:0/1423874544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f2038004c80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 -- 172.21.15.106:0/1423874544 >> 172.21.15.106:0/1423874544 conn(0x7f20540b2d50 msgr2=0x7f20540b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 -- 172.21.15.106:0/1423874544 shutdown_connections 2024-04-26T21:48:05.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.601+0000 7f2059e00640 1 -- 172.21.15.106:0/1423874544 wait complete. 2024-04-26T21:48:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.603+0000 7f2059e00640 1 Processor -- start 2024-04-26T21:48:05.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.604+0000 7f2059e00640 1 -- start start 2024-04-26T21:48:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2054072bf0 0x7f205415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 0x7f2054160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20540af0f0 con 0x7f2038004800 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20540aee10 con 0x7f2054073070 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2059e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f205415c600 con 0x7f2054072bf0 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2052a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2052a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46578/0 (socket says 172.21.15.106:46578) 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.605+0000 7f2052a00640 1 -- 172.21.15.106:0/2208413771 learned_addr learned my addr 172.21.15.106:0/2208413771 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052000640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2054072bf0 0x7f205415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2053400640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 0x7f2054160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 -- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2054072bf0 msgr2=0x7f205415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2054072bf0 0x7f205415fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 -- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 msgr2=0x7f2054160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 0x7f2054160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 -- 172.21.15.106:0/2208413771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20540735d0 con 0x7f2038004800 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2053400640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 0x7f2054160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f2052a00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20440116d0 tx=0x7f2044011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:05.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.606+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400f9c0 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.607+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204400fb60 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.607+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044015460 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.607+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20540738b0 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.607+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2054073e70 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.608+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2054069540 con 0x7f2038004800 2024-04-26T21:48:05.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.609+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2044015600 con 0x7f2038004800 2024-04-26T21:48:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.610+0000 7f204b400640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 0x7f203406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.610+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f204408c920 con 0x7f2038004800 2024-04-26T21:48:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.610+0000 7f2052000640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 0x7f203406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:05.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.611+0000 7f2052000640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 0x7f203406ef60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2040032000 tx=0x7f204002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:05.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.614+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2044019030 con 0x7f2038004800 2024-04-26T21:48:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.857+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20540cd430 con 0x7f2038004800 2024-04-26T21:48:05.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.858+0000 7f204b400640 1 -- 172.21.15.106:0/2208413771 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5190 (secure 0 0 0) 0x7f204405aac0 con 0x7f2038004800 2024-04-26T21:48:05.859 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:05.859 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:48:05.859 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 25 2024-04-26T21:48:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 msgr2=0x7f203406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 0x7f203406ef60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2040032000 tx=0x7f204002e000 comp rx=0 tx=0).stop 2024-04-26T21:48:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 msgr2=0x7f20540cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20440116d0 tx=0x7f2044011b90 comp rx=0 tx=0).stop 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 shutdown_connections 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2054073070 0x7f2054160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f203406caa0 0x7f203406ef60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2054072bf0 0x7f205415fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 --2- 172.21.15.106:0/2208413771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2038004800 0x7f20540cc6e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.861+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 >> 172.21.15.106:0/2208413771 conn(0x7f20540b2d50 msgr2=0x7f205406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.862+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 shutdown_connections 2024-04-26T21:48:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:05.862+0000 7f2059e00640 1 -- 172.21.15.106:0/2208413771 wait complete. 2024-04-26T21:48:05.881 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:06.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.081+0000 7fc52d600640 1 Processor -- start 2024-04-26T21:48:06.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.081+0000 7fc52d600640 1 -- start start 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc52d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52806c770 0x7fc52806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc52d600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc52815cb70 0x7fc528073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc52d600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52806d090 con 0x7fc52815cfd0 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc52d600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc5280af460 con 0x7fc52815cb70 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc52d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc5280ae850 con 0x7fc52806c770 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc526000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc52815cb70 0x7fc528073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc526000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc52815cb70 0x7fc528073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46586/0 (socket says 172.21.15.106:46586) 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc527400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc52815cfd0 0x7fc528073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36860/0 (socket says 172.21.15.106:36860) 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.083+0000 7fc526000640 1 -- 172.21.15.106:0/2673367030 learned_addr learned my addr 172.21.15.106:0/2673367030 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc526a00640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52806c770 0x7fc52806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc526000640 1 -- 172.21.15.106:0/2673367030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52806c770 msgr2=0x7fc52806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc526000640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52806c770 0x7fc52806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc526000640 1 -- 172.21.15.106:0/2673367030 >> v1:172.21.15.106:6789/0 conn(0x7fc52815cfd0 legacy=0x7fc528073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc525600640 1 -- 172.21.15.106:0/2673367030 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009031311 0 0) 0x7fc52806d090 con 0x7fc52815cfd0 2024-04-26T21:48:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.084+0000 7fc526000640 1 -- 172.21.15.106:0/2673367030 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5280aeb30 con 0x7fc52815cb70 2024-04-26T21:48:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.085+0000 7fc526000640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cb70 0x7fc528073120 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc510009870 tx=0x7fc51002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f4ad696f78db28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.085+0000 7fc525600640 1 -- 172.21.15.106:0/2673367030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510037070 con 0x7fc52815cb70 2024-04-26T21:48:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.085+0000 7fc525600640 1 -- 172.21.15.106:0/2673367030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510032440 con 0x7fc52815cb70 2024-04-26T21:48:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.085+0000 7fc525600640 1 -- 172.21.15.106:0/2673367030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510032810 con 0x7fc52815cb70 2024-04-26T21:48:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 -- 172.21.15.106:0/2673367030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cb70 msgr2=0x7fc528073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cb70 0x7fc528073120 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc510009870 tx=0x7fc51002a270 comp rx=0 tx=0).stop 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 -- 172.21.15.106:0/2673367030 shutdown_connections 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cb70 0x7fc528073120 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 --2- 172.21.15.106:0/2673367030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52806c770 0x7fc52806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 -- 172.21.15.106:0/2673367030 >> 172.21.15.106:0/2673367030 conn(0x7fc5280b2d50 msgr2=0x7fc5280b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 -- 172.21.15.106:0/2673367030 shutdown_connections 2024-04-26T21:48:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.086+0000 7fc52d600640 1 -- 172.21.15.106:0/2673367030 wait complete. 2024-04-26T21:48:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.088+0000 7fc52d600640 1 Processor -- start 2024-04-26T21:48:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.088+0000 7fc52d600640 1 -- start start 2024-04-26T21:48:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc52806c770 0x7fc5281f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 0x7fc52814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc52815c600 con 0x7fc52815cfd0 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5280af0f0 con 0x7fc52806c770 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc52d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc52815c780 con 0x7fc52815cb70 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc527400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc527400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46602/0 (socket says 172.21.15.106:46602) 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.089+0000 7fc527400640 1 -- 172.21.15.106:0/3778479884 learned_addr learned my addr 172.21.15.106:0/3778479884 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc526000640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 0x7fc52814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc526a00640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc52806c770 0x7fc5281f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 -- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 msgr2=0x7fc52814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 0x7fc52814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 -- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc52806c770 msgr2=0x7fc5281f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc52806c770 0x7fc5281f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 -- 172.21.15.106:0/3778479884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc528150020 con 0x7fc52815cfd0 2024-04-26T21:48:06.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc526000640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 0x7fc52814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:06.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc527400640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c700 tx=0x7fc52000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520018070 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52000ce10 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000a880 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.090+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281f5ff0 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.091+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281f8cc0 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.092+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc528069540 con 0x7fc52815cfd0 2024-04-26T21:48:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.093+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc52000aa20 con 0x7fc52815cfd0 2024-04-26T21:48:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.094+0000 7fc51ea00640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 0x7fc50406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.094+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc52008b500 con 0x7fc52815cfd0 2024-04-26T21:48:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.095+0000 7fc526a00640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 0x7fc50406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.095+0000 7fc526a00640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 0x7fc50406ef60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc518006fd0 tx=0x7fc518008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.099+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc520059610 con 0x7fc52815cfd0 2024-04-26T21:48:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.298+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc52806f890 con 0x7fc52815cfd0 2024-04-26T21:48:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.301+0000 7fc51ea00640 1 -- 172.21.15.106:0/3778479884 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16625 (secure 0 0 0) 0x7fc520058f10 con 0x7fc52815cfd0 2024-04-26T21:48:06.301 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:06.301 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":45,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:03.337384+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.303+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 msgr2=0x7fc50406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.303+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 0x7fc50406ef60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc518006fd0 tx=0x7fc518008040 comp rx=0 tx=0).stop 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.303+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 msgr2=0x7fc5281f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.303+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c700 tx=0x7fc52000cbc0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 shutdown_connections 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc52815cfd0 0x7fc5281f5460 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc50406caa0 0x7fc50406ef60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc52815cb70 0x7fc52814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 --2- 172.21.15.106:0/3778479884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc52806c770 0x7fc5281f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 >> 172.21.15.106:0/3778479884 conn(0x7fc5280b2d50 msgr2=0x7fc5280b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 shutdown_connections 2024-04-26T21:48:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.304+0000 7fc52d600640 1 -- 172.21.15.106:0/3778479884 wait complete. 2024-04-26T21:48:06.322 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:48:06.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:48:06.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.525+0000 7fefcf200640 1 Processor -- start 2024-04-26T21:48:06.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.525+0000 7fefcf200640 1 -- start start 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefcf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 0x7fefc80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefcf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefcf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc80af9a0 con 0x7fefc80b4f50 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefcf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fefc80ae2c0 con 0x7fefc80cd650 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefcf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefc80b05b0 con 0x7fefc815cdf0 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefbf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.527+0000 7fefc7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 0x7fefc80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefbf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:34888/0 (socket says 172.21.15.106:34888) 2024-04-26T21:48:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefbf400640 1 -- 172.21.15.106:0/282034787 learned_addr learned my addr 172.21.15.106:0/282034787 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefc7e00640 1 -- 172.21.15.106:0/282034787 >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 msgr2=0x7fefc806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefc7e00640 1 --2- 172.21.15.106:0/282034787 >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefc7e00640 1 -- 172.21.15.106:0/282034787 >> v1:172.21.15.204:6789/0 conn(0x7fefc80b4f50 legacy=0x7fefc8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefc7e00640 1 -- 172.21.15.106:0/282034787 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc80afc80 con 0x7fefc815cdf0 2024-04-26T21:48:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.528+0000 7fefbf400640 1 --2- 172.21.15.106:0/282034787 >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:48:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.530+0000 7fefc7e00640 1 --2- 172.21.15.106:0/282034787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 0x7fefc80add80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c3a3969d45b124 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefc7400640 1 -- 172.21.15.106:0/282034787 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8037070 con 0x7fefc815cdf0 2024-04-26T21:48:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefc7400640 1 -- 172.21.15.106:0/282034787 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8032440 con 0x7fefc815cdf0 2024-04-26T21:48:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefc7400640 1 -- 172.21.15.106:0/282034787 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8032810 con 0x7fefc815cdf0 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 -- 172.21.15.106:0/282034787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 msgr2=0x7fefc80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 --2- 172.21.15.106:0/282034787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 0x7fefc80add80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802a180 comp rx=0 tx=0).stop 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 -- 172.21.15.106:0/282034787 shutdown_connections 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 --2- 172.21.15.106:0/282034787 >> v2:172.21.15.204:3300/0 conn(0x7fefc80cd650 0x7fefc806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 --2- 172.21.15.106:0/282034787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc815cdf0 0x7fefc80add80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 -- 172.21.15.106:0/282034787 >> 172.21.15.106:0/282034787 conn(0x7fefc80b2d50 msgr2=0x7fefc80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:06.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 -- 172.21.15.106:0/282034787 shutdown_connections 2024-04-26T21:48:06.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.531+0000 7fefcf200640 1 -- 172.21.15.106:0/282034787 wait complete. 2024-04-26T21:48:06.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.533+0000 7fefcf200640 1 Processor -- start 2024-04-26T21:48:06.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.533+0000 7fefcf200640 1 -- start start 2024-04-26T21:48:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc80b4f50 0x7fefc809d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc80cd650 0x7fefc809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fefc815d1d0 con 0x7fefc815cdf0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fefc80b0240 con 0x7fefc80b4f50 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefc815d350 con 0x7fefc80cd650 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.534+0000 7fefcd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46612/0 (socket says 172.21.15.106:46612) 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 -- 172.21.15.106:0/2841799382 learned_addr learned my addr 172.21.15.106:0/2841799382 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefc7e00640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc80b4f50 0x7fefc809d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefbf400640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc80cd650 0x7fefc809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 -- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc80cd650 msgr2=0x7fefc809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc80cd650 0x7fefc809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 -- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc80b4f50 msgr2=0x7fefc809d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc80b4f50 0x7fefc809d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 -- 172.21.15.106:0/2841799382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc81f48e0 con 0x7fefc815cdf0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefcd200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c6e0 tx=0x7fefc000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.535+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0003b70 con 0x7fefc815cdf0 2024-04-26T21:48:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.536+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0003d30 con 0x7fefc815cdf0 2024-04-26T21:48:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.536+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0005310 con 0x7fefc815cdf0 2024-04-26T21:48:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.536+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc81f83b0 con 0x7fefc815cdf0 2024-04-26T21:48:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.536+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc81f8970 con 0x7fefc815cdf0 2024-04-26T21:48:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.537+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc8069540 con 0x7fefc815cdf0 2024-04-26T21:48:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.538+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fefc00054b0 con 0x7fefc815cdf0 2024-04-26T21:48:06.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.539+0000 7fefc4c00640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 0x7fefa406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:06.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.539+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fefc008be30 con 0x7fefc815cdf0 2024-04-26T21:48:06.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.539+0000 7fefc7e00640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 0x7fefa406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:06.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.540+0000 7fefc7e00640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 0x7fefa406ef90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefb8034000 tx=0x7fefb802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:06.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.543+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefc81f48e0 con 0x7fefc815cdf0 2024-04-26T21:48:06.690 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-26T21:48:06.690+0000 7f349f200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:48:06.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.725+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefc80008d0 con 0x7fefa406cad0 2024-04-26T21:48:06.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.744+0000 7fefc4c00640 1 -- 172.21.15.106:0/2841799382 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7fefc80008d0 con 0x7fefa406cad0 2024-04-26T21:48:06.747 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:06.748 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 msgr2=0x7fefa406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 0x7fefa406ef90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefb8034000 tx=0x7fefb802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 msgr2=0x7fefc80cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c6e0 tx=0x7fefc000cba0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 shutdown_connections 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefc815cdf0 0x7fefc80cf1c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fefa406cad0 0x7fefa406ef90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefc80cd650 0x7fefc809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 --2- 172.21.15.106:0/2841799382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc80b4f50 0x7fefc809d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 >> 172.21.15.106:0/2841799382 conn(0x7fefc80b2d50 msgr2=0x7fefc80b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 shutdown_connections 2024-04-26T21:48:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:06.749+0000 7fefcf200640 1 -- 172.21.15.106:0/2841799382 wait complete. 2024-04-26T21:48:06.767 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-04-26T21:48:05.128517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229424,"kb_used_data":4904,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745296,"statfs":{"total":773094113280,"available":772859183104,"internally_reserved":0,"allocated":5021696,"data_stored":2700384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819268"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:01.307732+0000","last_change":"2024-04-26T21:48:01.307732+0000","last_active":"2024-04-26T21:48:01.307732+0000","last_peered":"2024-04-26T21:48:01.307732+0000","last_clean":"2024-04-26T21:48:01.307732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:01.307732+0000","last_undegraded":"2024-04-26T21:48:01.307732+0000","last_fullsized":"2024-04-26T21:48:01.307732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:48:00.292082+0000","last_change":"2024-04-26T21:48:00.292082+0000","last_active":"2024-04-26T21:48:00.292082+0000","last_peered":"2024-04-26T21:48:00.292082+0000","last_clean":"2024-04-26T21:48:00.292082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:48:00.292082+0000","last_undegraded":"2024-04-26T21:48:00.292082+0000","last_fullsized":"2024-04-26T21:48:00.292082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:47:57.284949+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:47:57.284949+0000","last_peered":"2024-04-26T21:47:57.284949+0000","last_clean":"2024-04-26T21:47:57.284949+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:47:57.284949+0000","last_undegraded":"2024-04-26T21:47:57.284949+0000","last_fullsized":"2024-04-26T21:47:57.284949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1073152,"data_stored":780324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":780324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:48:06.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:48:07.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.007+0000 7f7d07e00640 1 Processor -- start 2024-04-26T21:48:07.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.008+0000 7f7d07e00640 1 -- start start 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 0x7f7d080d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d08162b00 0x7f7d08162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d08163910 con 0x7f7d08163480 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7d080d3f50 con 0x7f7d08162b00 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080d36b0 con 0x7f7d0815f1b0 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d06000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d08162b00 0x7f7d08162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7d08163480 0x7f7d080cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36872/0 (socket says 172.21.15.106:36872) 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.009+0000 7f7d07400640 1 -- 172.21.15.106:0/1032075258 learned_addr learned my addr 172.21.15.106:0/1032075258 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06a00640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 0x7f7d080d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06000640 1 -- 172.21.15.106:0/1032075258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 msgr2=0x7f7d080d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:07.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06000640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 0x7f7d080d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06000640 1 -- 172.21.15.106:0/1032075258 >> v1:172.21.15.106:6789/0 conn(0x7f7d08163480 legacy=0x7f7d080cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06000640 1 -- 172.21.15.106:0/1032075258 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d080d3990 con 0x7f7d08162b00 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.010+0000 7f7d06a00640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 0x7f7d080d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.011+0000 7f7d06000640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08162b00 0x7f7d08162f40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00097a0 tx=0x7f7cf002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae435c0ed7cad65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.011+0000 7f7d05600640 1 -- 172.21.15.106:0/1032075258 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037070 con 0x7f7d08162b00 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.011+0000 7f7d05600640 1 -- 172.21.15.106:0/1032075258 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032440 con 0x7f7d08162b00 2024-04-26T21:48:07.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.011+0000 7f7d05600640 1 -- 172.21.15.106:0/1032075258 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0032810 con 0x7f7d08162b00 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 -- 172.21.15.106:0/1032075258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08162b00 msgr2=0x7f7d08162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08162b00 0x7f7d08162f40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00097a0 tx=0x7f7cf002a180 comp rx=0 tx=0).stop 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 -- 172.21.15.106:0/1032075258 shutdown_connections 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08162b00 0x7f7d08162f40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 --2- 172.21.15.106:0/1032075258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d0815f1b0 0x7f7d080d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 -- 172.21.15.106:0/1032075258 >> 172.21.15.106:0/1032075258 conn(0x7f7d0809e230 msgr2=0x7f7d080d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:07.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 -- 172.21.15.106:0/1032075258 shutdown_connections 2024-04-26T21:48:07.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.012+0000 7f7d07e00640 1 -- 172.21.15.106:0/1032075258 wait complete. 2024-04-26T21:48:07.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.014+0000 7f7d07e00640 1 Processor -- start 2024-04-26T21:48:07.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.014+0000 7f7d07e00640 1 -- start start 2024-04-26T21:48:07.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d08162b00 0x7f7d080cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d08163480 0x7f7d0815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080d33a0 con 0x7f7d0815f1b0 2024-04-26T21:48:07.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080b7d90 con 0x7f7d08162b00 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d07e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080b92c0 con 0x7f7d08163480 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46632/0 (socket says 172.21.15.106:46632) 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 -- 172.21.15.106:0/728858958 learned_addr learned my addr 172.21.15.106:0/728858958 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 -- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d08163480 msgr2=0x7f7d0815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d08163480 0x7f7d0815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 -- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d08162b00 msgr2=0x7f7d080cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d08162b00 0x7f7d080cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.015+0000 7f7d06a00640 1 -- 172.21.15.106:0/728858958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0814d640 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7d06a00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c820 tx=0x7f7cf800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8015070 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf8007a30 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8007e00 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d08161f90 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.016+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d080cdd20 con 0x7f7d0815f1b0 2024-04-26T21:48:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.017+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080693c0 con 0x7f7d0815f1b0 2024-04-26T21:48:07.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.018+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f7cf8018070 con 0x7f7d0815f1b0 2024-04-26T21:48:07.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.019+0000 7f7cfea00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 0x7f7ce406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.019+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf808bf80 con 0x7f7d0815f1b0 2024-04-26T21:48:07.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.020+0000 7f7d06000640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 0x7f7ce406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.020+0000 7f7d06000640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 0x7f7ce406ef60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.024+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7cf805a010 con 0x7f7d0815f1b0 2024-04-26T21:48:07.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.221+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d0808d090 con 0x7f7ce406caa0 2024-04-26T21:48:07.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.232+0000 7f7cfea00640 1 -- 172.21.15.106:0/728858958 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f7d0808d090 con 0x7f7ce406caa0 2024-04-26T21:48:07.236 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:07.237 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 msgr2=0x7f7ce406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 0x7f7ce406ef60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002c000 comp rx=0 tx=0).stop 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 msgr2=0x7f7d0814d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c820 tx=0x7f7cf800cc20 comp rx=0 tx=0).stop 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 shutdown_connections 2024-04-26T21:48:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7ce406caa0 0x7f7ce406ef60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d08163480 0x7f7d0815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d08162b00 0x7f7d080cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.237+0000 7f7d07e00640 1 --2- 172.21.15.106:0/728858958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815f1b0 0x7f7d0814d100 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.238+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 >> 172.21.15.106:0/728858958 conn(0x7f7d0809e230 msgr2=0x7f7d0806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.238+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 shutdown_connections 2024-04-26T21:48:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.238+0000 7f7d07e00640 1 -- 172.21.15.106:0/728858958 wait complete. 2024-04-26T21:48:07.257 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-26T21:48:07.129072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229852,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744868,"statfs":{"total":773094113280,"available":772858744832,"internally_reserved":0,"allocated":5480448,"data_stored":2906476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001715"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360550+0000","last_change":"2024-04-26T21:48:02.353031+0000","last_active":"2024-04-26T21:48:03.360550+0000","last_peered":"2024-04-26T21:48:03.360550+0000","last_clean":"2024-04-26T21:48:03.360550+0000","last_became_active":"2024-04-26T21:48:02.352715+0000","last_became_peered":"2024-04-26T21:48:02.352715+0000","last_unstale":"2024-04-26T21:48:03.360550+0000","last_undegraded":"2024-04-26T21:48:03.360550+0000","last_fullsized":"2024-04-26T21:48:03.360550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:17.105683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.362039+0000","last_change":"2024-04-26T21:48:02.359991+0000","last_active":"2024-04-26T21:48:03.362039+0000","last_peered":"2024-04-26T21:48:03.362039+0000","last_clean":"2024-04-26T21:48:03.362039+0000","last_became_active":"2024-04-26T21:48:02.359660+0000","last_became_peered":"2024-04-26T21:48:02.359660+0000","last_unstale":"2024-04-26T21:48:03.362039+0000","last_undegraded":"2024-04-26T21:48:03.362039+0000","last_fullsized":"2024-04-26T21:48:03.362039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:58.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.378812+0000","last_change":"2024-04-26T21:48:02.344824+0000","last_active":"2024-04-26T21:48:03.378812+0000","last_peered":"2024-04-26T21:48:03.378812+0000","last_clean":"2024-04-26T21:48:03.378812+0000","last_became_active":"2024-04-26T21:48:02.344502+0000","last_became_peered":"2024-04-26T21:48:02.344502+0000","last_unstale":"2024-04-26T21:48:03.378812+0000","last_undegraded":"2024-04-26T21:48:03.378812+0000","last_fullsized":"2024-04-26T21:48:03.378812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:23.311329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778715+0000","last_change":"2024-04-26T21:48:02.358067+0000","last_active":"2024-04-26T21:48:03.778715+0000","last_peered":"2024-04-26T21:48:03.778715+0000","last_clean":"2024-04-26T21:48:03.778715+0000","last_became_active":"2024-04-26T21:48:02.357751+0000","last_became_peered":"2024-04-26T21:48:02.357751+0000","last_unstale":"2024-04-26T21:48:03.778715+0000","last_undegraded":"2024-04-26T21:48:03.778715+0000","last_fullsized":"2024-04-26T21:48:03.778715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:17.105683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360964+0000","last_change":"2024-04-26T21:48:02.363879+0000","last_active":"2024-04-26T21:48:03.360964+0000","last_peered":"2024-04-26T21:48:03.360964+0000","last_clean":"2024-04-26T21:48:03.360964+0000","last_became_active":"2024-04-26T21:48:02.362942+0000","last_became_peered":"2024-04-26T21:48:02.362942+0000","last_unstale":"2024-04-26T21:48:03.360964+0000","last_undegraded":"2024-04-26T21:48:03.360964+0000","last_fullsized":"2024-04-26T21:48:03.360964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:20.518916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372881+0000","last_change":"2024-04-26T21:48:02.356388+0000","last_active":"2024-04-26T21:48:03.372881+0000","last_peered":"2024-04-26T21:48:03.372881+0000","last_clean":"2024-04-26T21:48:03.372881+0000","last_became_active":"2024-04-26T21:48:02.355859+0000","last_became_peered":"2024-04-26T21:48:02.355859+0000","last_unstale":"2024-04-26T21:48:03.372881+0000","last_undegraded":"2024-04-26T21:48:03.372881+0000","last_fullsized":"2024-04-26T21:48:03.372881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:34.562990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.359680+0000","last_change":"2024-04-26T21:48:02.352592+0000","last_active":"2024-04-26T21:48:03.359680+0000","last_peered":"2024-04-26T21:48:03.359680+0000","last_clean":"2024-04-26T21:48:03.359680+0000","last_became_active":"2024-04-26T21:48:02.352313+0000","last_became_peered":"2024-04-26T21:48:02.352313+0000","last_unstale":"2024-04-26T21:48:03.359680+0000","last_undegraded":"2024-04-26T21:48:03.359680+0000","last_fullsized":"2024-04-26T21:48:03.359680+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:21.682442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777324+0000","last_change":"2024-04-26T21:48:02.360464+0000","last_active":"2024-04-26T21:48:03.777324+0000","last_peered":"2024-04-26T21:48:03.777324+0000","last_clean":"2024-04-26T21:48:03.777324+0000","last_became_active":"2024-04-26T21:48:02.360067+0000","last_became_peered":"2024-04-26T21:48:02.360067+0000","last_unstale":"2024-04-26T21:48:03.777324+0000","last_undegraded":"2024-04-26T21:48:03.777324+0000","last_fullsized":"2024-04-26T21:48:03.777324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:54.899019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778974+0000","last_change":"2024-04-26T21:48:02.359803+0000","last_active":"2024-04-26T21:48:03.778974+0000","last_peered":"2024-04-26T21:48:03.778974+0000","last_clean":"2024-04-26T21:48:03.778974+0000","last_became_active":"2024-04-26T21:48:02.359549+0000","last_became_peered":"2024-04-26T21:48:02.359549+0000","last_unstale":"2024-04-26T21:48:03.778974+0000","last_undegraded":"2024-04-26T21:48:03.778974+0000","last_fullsized":"2024-04-26T21:48:03.778974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:25.946408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778924+0000","last_change":"2024-04-26T21:48:02.362818+0000","last_active":"2024-04-26T21:48:03.778924+0000","last_peered":"2024-04-26T21:48:03.778924+0000","last_clean":"2024-04-26T21:48:03.778924+0000","last_became_active":"2024-04-26T21:48:02.362446+0000","last_became_peered":"2024-04-26T21:48:02.362446+0000","last_unstale":"2024-04-26T21:48:03.778924+0000","last_undegraded":"2024-04-26T21:48:03.778924+0000","last_fullsized":"2024-04-26T21:48:03.778924+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:20.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778698+0000","last_change":"2024-04-26T21:48:02.364339+0000","last_active":"2024-04-26T21:48:03.778698+0000","last_peered":"2024-04-26T21:48:03.778698+0000","last_clean":"2024-04-26T21:48:03.778698+0000","last_became_active":"2024-04-26T21:48:02.363833+0000","last_became_peered":"2024-04-26T21:48:02.363833+0000","last_unstale":"2024-04-26T21:48:03.778698+0000","last_undegraded":"2024-04-26T21:48:03.778698+0000","last_fullsized":"2024-04-26T21:48:03.778698+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:58.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361535+0000","last_change":"2024-04-26T21:48:02.362974+0000","last_active":"2024-04-26T21:48:03.361535+0000","last_peered":"2024-04-26T21:48:03.361535+0000","last_clean":"2024-04-26T21:48:03.361535+0000","last_became_active":"2024-04-26T21:48:02.362681+0000","last_became_peered":"2024-04-26T21:48:02.362681+0000","last_unstale":"2024-04-26T21:48:03.361535+0000","last_undegraded":"2024-04-26T21:48:03.361535+0000","last_fullsized":"2024-04-26T21:48:03.361535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:16.481924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361318+0000","last_change":"2024-04-26T21:48:02.352447+0000","last_active":"2024-04-26T21:48:03.361318+0000","last_peered":"2024-04-26T21:48:03.361318+0000","last_clean":"2024-04-26T21:48:03.361318+0000","last_became_active":"2024-04-26T21:48:02.352147+0000","last_became_peered":"2024-04-26T21:48:02.352147+0000","last_unstale":"2024-04-26T21:48:03.361318+0000","last_undegraded":"2024-04-26T21:48:03.361318+0000","last_fullsized":"2024-04-26T21:48:03.361318+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:56.730993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778084+0000","last_change":"2024-04-26T21:48:02.359190+0000","last_active":"2024-04-26T21:48:03.778084+0000","last_peered":"2024-04-26T21:48:03.778084+0000","last_clean":"2024-04-26T21:48:03.778084+0000","last_became_active":"2024-04-26T21:48:02.358932+0000","last_became_peered":"2024-04-26T21:48:02.358932+0000","last_unstale":"2024-04-26T21:48:03.778084+0000","last_undegraded":"2024-04-26T21:48:03.778084+0000","last_fullsized":"2024-04-26T21:48:03.778084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:13.951053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372682+0000","last_change":"2024-04-26T21:48:02.354160+0000","last_active":"2024-04-26T21:48:03.372682+0000","last_peered":"2024-04-26T21:48:03.372682+0000","last_clean":"2024-04-26T21:48:03.372682+0000","last_became_active":"2024-04-26T21:48:02.353849+0000","last_became_peered":"2024-04-26T21:48:02.353849+0000","last_unstale":"2024-04-26T21:48:03.372682+0000","last_undegraded":"2024-04-26T21:48:03.372682+0000","last_fullsized":"2024-04-26T21:48:03.372682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:47.266169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090164+0000","last_change":"2024-04-26T21:48:02.364686+0000","last_active":"2024-04-26T21:48:04.090164+0000","last_peered":"2024-04-26T21:48:04.090164+0000","last_clean":"2024-04-26T21:48:04.090164+0000","last_became_active":"2024-04-26T21:48:02.364151+0000","last_became_peered":"2024-04-26T21:48:02.364151+0000","last_unstale":"2024-04-26T21:48:04.090164+0000","last_undegraded":"2024-04-26T21:48:04.090164+0000","last_fullsized":"2024-04-26T21:48:04.090164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:48.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379225+0000","last_change":"2024-04-26T21:48:02.346224+0000","last_active":"2024-04-26T21:48:03.379225+0000","last_peered":"2024-04-26T21:48:03.379225+0000","last_clean":"2024-04-26T21:48:03.379225+0000","last_became_active":"2024-04-26T21:48:02.345951+0000","last_became_peered":"2024-04-26T21:48:02.345951+0000","last_unstale":"2024-04-26T21:48:03.379225+0000","last_undegraded":"2024-04-26T21:48:03.379225+0000","last_fullsized":"2024-04-26T21:48:03.379225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:00.350196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379324+0000","last_change":"2024-04-26T21:48:02.358244+0000","last_active":"2024-04-26T21:48:03.379324+0000","last_peered":"2024-04-26T21:48:03.379324+0000","last_clean":"2024-04-26T21:48:03.379324+0000","last_became_active":"2024-04-26T21:48:02.357081+0000","last_became_peered":"2024-04-26T21:48:02.357081+0000","last_unstale":"2024-04-26T21:48:03.379324+0000","last_undegraded":"2024-04-26T21:48:03.379324+0000","last_fullsized":"2024-04-26T21:48:03.379324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:54.899019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090572+0000","last_change":"2024-04-26T21:48:02.368480+0000","last_active":"2024-04-26T21:48:04.090572+0000","last_peered":"2024-04-26T21:48:04.090572+0000","last_clean":"2024-04-26T21:48:04.090572+0000","last_became_active":"2024-04-26T21:48:02.368246+0000","last_became_peered":"2024-04-26T21:48:02.368246+0000","last_unstale":"2024-04-26T21:48:04.090572+0000","last_undegraded":"2024-04-26T21:48:04.090572+0000","last_fullsized":"2024-04-26T21:48:04.090572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:00.752368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361072+0000","last_change":"2024-04-26T21:48:02.364974+0000","last_active":"2024-04-26T21:48:03.361072+0000","last_peered":"2024-04-26T21:48:03.361072+0000","last_clean":"2024-04-26T21:48:03.361072+0000","last_became_active":"2024-04-26T21:48:02.364589+0000","last_became_peered":"2024-04-26T21:48:02.364589+0000","last_unstale":"2024-04-26T21:48:03.361072+0000","last_undegraded":"2024-04-26T21:48:03.361072+0000","last_fullsized":"2024-04-26T21:48:03.361072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:15.402095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372589+0000","last_change":"2024-04-26T21:48:02.354673+0000","last_active":"2024-04-26T21:48:03.372589+0000","last_peered":"2024-04-26T21:48:03.372589+0000","last_clean":"2024-04-26T21:48:03.372589+0000","last_became_active":"2024-04-26T21:48:02.354415+0000","last_became_peered":"2024-04-26T21:48:02.354415+0000","last_unstale":"2024-04-26T21:48:03.372589+0000","last_undegraded":"2024-04-26T21:48:03.372589+0000","last_fullsized":"2024-04-26T21:48:03.372589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:42.726913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777793+0000","last_change":"2024-04-26T21:48:02.354755+0000","last_active":"2024-04-26T21:48:03.777793+0000","last_peered":"2024-04-26T21:48:03.777793+0000","last_clean":"2024-04-26T21:48:03.777793+0000","last_became_active":"2024-04-26T21:48:02.354454+0000","last_became_peered":"2024-04-26T21:48:02.354454+0000","last_unstale":"2024-04-26T21:48:03.777793+0000","last_undegraded":"2024-04-26T21:48:03.777793+0000","last_fullsized":"2024-04-26T21:48:03.777793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:16.481924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089888+0000","last_change":"2024-04-26T21:48:02.353812+0000","last_active":"2024-04-26T21:48:04.089888+0000","last_peered":"2024-04-26T21:48:04.089888+0000","last_clean":"2024-04-26T21:48:04.089888+0000","last_became_active":"2024-04-26T21:48:02.350698+0000","last_became_peered":"2024-04-26T21:48:02.350698+0000","last_unstale":"2024-04-26T21:48:04.089888+0000","last_undegraded":"2024-04-26T21:48:04.089888+0000","last_fullsized":"2024-04-26T21:48:04.089888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:48.620840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090311+0000","last_change":"2024-04-26T21:48:02.356695+0000","last_active":"2024-04-26T21:48:04.090311+0000","last_peered":"2024-04-26T21:48:04.090311+0000","last_clean":"2024-04-26T21:48:04.090311+0000","last_became_active":"2024-04-26T21:48:02.355091+0000","last_became_peered":"2024-04-26T21:48:02.355091+0000","last_unstale":"2024-04-26T21:48:04.090311+0000","last_undegraded":"2024-04-26T21:48:04.090311+0000","last_fullsized":"2024-04-26T21:48:04.090311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:48.253954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090390+0000","last_change":"2024-04-26T21:48:02.360261+0000","last_active":"2024-04-26T21:48:04.090390+0000","last_peered":"2024-04-26T21:48:04.090390+0000","last_clean":"2024-04-26T21:48:04.090390+0000","last_became_active":"2024-04-26T21:48:02.358837+0000","last_became_peered":"2024-04-26T21:48:02.358837+0000","last_unstale":"2024-04-26T21:48:04.090390+0000","last_undegraded":"2024-04-26T21:48:04.090390+0000","last_fullsized":"2024-04-26T21:48:04.090390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:48.604553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360805+0000","last_change":"2024-04-26T21:48:02.351942+0000","last_active":"2024-04-26T21:48:03.360805+0000","last_peered":"2024-04-26T21:48:03.360805+0000","last_clean":"2024-04-26T21:48:03.360805+0000","last_became_active":"2024-04-26T21:48:02.351668+0000","last_became_peered":"2024-04-26T21:48:02.351668+0000","last_unstale":"2024-04-26T21:48:03.360805+0000","last_undegraded":"2024-04-26T21:48:03.360805+0000","last_fullsized":"2024-04-26T21:48:03.360805+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:26.927630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379065+0000","last_change":"2024-04-26T21:48:02.348446+0000","last_active":"2024-04-26T21:48:03.379065+0000","last_peered":"2024-04-26T21:48:03.379065+0000","last_clean":"2024-04-26T21:48:03.379065+0000","last_became_active":"2024-04-26T21:48:02.348135+0000","last_became_peered":"2024-04-26T21:48:02.348135+0000","last_unstale":"2024-04-26T21:48:03.379065+0000","last_undegraded":"2024-04-26T21:48:03.379065+0000","last_fullsized":"2024-04-26T21:48:03.379065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:51.844776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361952+0000","last_change":"2024-04-26T21:48:02.353120+0000","last_active":"2024-04-26T21:48:03.361952+0000","last_peered":"2024-04-26T21:48:03.361952+0000","last_clean":"2024-04-26T21:48:03.361952+0000","last_became_active":"2024-04-26T21:48:02.351400+0000","last_became_peered":"2024-04-26T21:48:02.351400+0000","last_unstale":"2024-04-26T21:48:03.361952+0000","last_undegraded":"2024-04-26T21:48:03.361952+0000","last_fullsized":"2024-04-26T21:48:03.361952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:21.682442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089827+0000","last_change":"2024-04-26T21:48:02.358618+0000","last_active":"2024-04-26T21:48:04.089827+0000","last_peered":"2024-04-26T21:48:04.089827+0000","last_clean":"2024-04-26T21:48:04.089827+0000","last_became_active":"2024-04-26T21:48:02.358208+0000","last_became_peered":"2024-04-26T21:48:02.358208+0000","last_unstale":"2024-04-26T21:48:04.089827+0000","last_undegraded":"2024-04-26T21:48:04.089827+0000","last_fullsized":"2024-04-26T21:48:04.089827+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:41.055200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.378977+0000","last_change":"2024-04-26T21:48:02.356508+0000","last_active":"2024-04-26T21:48:03.378977+0000","last_peered":"2024-04-26T21:48:03.378977+0000","last_clean":"2024-04-26T21:48:03.378977+0000","last_became_active":"2024-04-26T21:48:02.356190+0000","last_became_peered":"2024-04-26T21:48:02.356190+0000","last_unstale":"2024-04-26T21:48:03.378977+0000","last_undegraded":"2024-04-26T21:48:03.378977+0000","last_fullsized":"2024-04-26T21:48:03.378977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:00.518536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372718+0000","last_change":"2024-04-26T21:48:02.347092+0000","last_active":"2024-04-26T21:48:03.372718+0000","last_peered":"2024-04-26T21:48:03.372718+0000","last_clean":"2024-04-26T21:48:03.372718+0000","last_became_active":"2024-04-26T21:48:02.346782+0000","last_became_peered":"2024-04-26T21:48:02.346782+0000","last_unstale":"2024-04-26T21:48:03.372718+0000","last_undegraded":"2024-04-26T21:48:03.372718+0000","last_fullsized":"2024-04-26T21:48:03.372718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:59.797690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361289+0000","last_change":"2024-04-26T21:48:02.347170+0000","last_active":"2024-04-26T21:48:03.361289+0000","last_peered":"2024-04-26T21:48:03.361289+0000","last_clean":"2024-04-26T21:48:03.361289+0000","last_became_active":"2024-04-26T21:48:02.346712+0000","last_became_peered":"2024-04-26T21:48:02.346712+0000","last_unstale":"2024-04-26T21:48:03.361289+0000","last_undegraded":"2024-04-26T21:48:03.361289+0000","last_fullsized":"2024-04-26T21:48:03.361289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:43.735411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090091+0000","last_change":"2024-04-26T21:48:02.349712+0000","last_active":"2024-04-26T21:48:04.090091+0000","last_peered":"2024-04-26T21:48:04.090091+0000","last_clean":"2024-04-26T21:48:04.090091+0000","last_became_active":"2024-04-26T21:48:02.349374+0000","last_became_peered":"2024-04-26T21:48:02.349374+0000","last_unstale":"2024-04-26T21:48:04.090091+0000","last_undegraded":"2024-04-26T21:48:04.090091+0000","last_fullsized":"2024-04-26T21:48:04.090091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:37.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373950+0000","last_change":"2024-04-26T21:48:01.365149+0000","last_active":"2024-04-26T21:48:03.373950+0000","last_peered":"2024-04-26T21:48:03.373950+0000","last_clean":"2024-04-26T21:48:03.373950+0000","last_became_active":"2024-04-26T21:48:01.364821+0000","last_became_peered":"2024-04-26T21:48:01.364821+0000","last_unstale":"2024-04-26T21:48:03.373950+0000","last_undegraded":"2024-04-26T21:48:03.373950+0000","last_fullsized":"2024-04-26T21:48:03.373950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:50.829126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090664+0000","last_change":"2024-04-26T21:48:02.353367+0000","last_active":"2024-04-26T21:48:04.090664+0000","last_peered":"2024-04-26T21:48:04.090664+0000","last_clean":"2024-04-26T21:48:04.090664+0000","last_became_active":"2024-04-26T21:48:02.352328+0000","last_became_peered":"2024-04-26T21:48:02.352328+0000","last_unstale":"2024-04-26T21:48:04.090664+0000","last_undegraded":"2024-04-26T21:48:04.090664+0000","last_fullsized":"2024-04-26T21:48:04.090664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:37.638059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379516+0000","last_change":"2024-04-26T21:48:01.356244+0000","last_active":"2024-04-26T21:48:03.379516+0000","last_peered":"2024-04-26T21:48:03.379516+0000","last_clean":"2024-04-26T21:48:03.379516+0000","last_became_active":"2024-04-26T21:48:01.355876+0000","last_became_peered":"2024-04-26T21:48:01.355876+0000","last_unstale":"2024-04-26T21:48:03.379516+0000","last_undegraded":"2024-04-26T21:48:03.379516+0000","last_fullsized":"2024-04-26T21:48:03.379516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:47.605190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.362020+0000","last_change":"2024-04-26T21:48:02.354403+0000","last_active":"2024-04-26T21:48:03.362020+0000","last_peered":"2024-04-26T21:48:03.362020+0000","last_clean":"2024-04-26T21:48:03.362020+0000","last_became_active":"2024-04-26T21:48:02.351120+0000","last_became_peered":"2024-04-26T21:48:02.351120+0000","last_unstale":"2024-04-26T21:48:03.362020+0000","last_undegraded":"2024-04-26T21:48:03.362020+0000","last_fullsized":"2024-04-26T21:48:03.362020+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:13.951053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090144+0000","last_change":"2024-04-26T21:48:01.362922+0000","last_active":"2024-04-26T21:48:04.090144+0000","last_peered":"2024-04-26T21:48:04.090144+0000","last_clean":"2024-04-26T21:48:04.090144+0000","last_became_active":"2024-04-26T21:48:01.362679+0000","last_became_peered":"2024-04-26T21:48:01.362679+0000","last_unstale":"2024-04-26T21:48:04.090144+0000","last_undegraded":"2024-04-26T21:48:04.090144+0000","last_fullsized":"2024-04-26T21:48:04.090144+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.238304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090151+0000","last_change":"2024-04-26T21:48:02.351264+0000","last_active":"2024-04-26T21:48:04.090151+0000","last_peered":"2024-04-26T21:48:04.090151+0000","last_clean":"2024-04-26T21:48:04.090151+0000","last_became_active":"2024-04-26T21:48:02.351007+0000","last_became_peered":"2024-04-26T21:48:02.351007+0000","last_unstale":"2024-04-26T21:48:04.090151+0000","last_undegraded":"2024-04-26T21:48:04.090151+0000","last_fullsized":"2024-04-26T21:48:04.090151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:07.758660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.359051+0000","last_change":"2024-04-26T21:48:01.364066+0000","last_active":"2024-04-26T21:48:03.359051+0000","last_peered":"2024-04-26T21:48:03.359051+0000","last_clean":"2024-04-26T21:48:03.359051+0000","last_became_active":"2024-04-26T21:48:01.363743+0000","last_became_peered":"2024-04-26T21:48:01.363743+0000","last_unstale":"2024-04-26T21:48:03.359051+0000","last_undegraded":"2024-04-26T21:48:03.359051+0000","last_fullsized":"2024-04-26T21:48:03.359051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:06.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.378795+0000","last_change":"2024-04-26T21:48:02.350122+0000","last_active":"2024-04-26T21:48:03.378795+0000","last_peered":"2024-04-26T21:48:03.378795+0000","last_clean":"2024-04-26T21:48:03.378795+0000","last_became_active":"2024-04-26T21:48:02.348642+0000","last_became_peered":"2024-04-26T21:48:02.348642+0000","last_unstale":"2024-04-26T21:48:03.378795+0000","last_undegraded":"2024-04-26T21:48:03.378795+0000","last_fullsized":"2024-04-26T21:48:03.378795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:13.933431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372960+0000","last_change":"2024-04-26T21:48:01.362153+0000","last_active":"2024-04-26T21:48:03.372960+0000","last_peered":"2024-04-26T21:48:03.372960+0000","last_clean":"2024-04-26T21:48:03.372960+0000","last_became_active":"2024-04-26T21:48:01.361907+0000","last_became_peered":"2024-04-26T21:48:01.361907+0000","last_unstale":"2024-04-26T21:48:03.372960+0000","last_undegraded":"2024-04-26T21:48:03.372960+0000","last_fullsized":"2024-04-26T21:48:03.372960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:15.466275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777690+0000","last_change":"2024-04-26T21:48:02.368060+0000","last_active":"2024-04-26T21:48:03.777690+0000","last_peered":"2024-04-26T21:48:03.777690+0000","last_clean":"2024-04-26T21:48:03.777690+0000","last_became_active":"2024-04-26T21:48:02.367691+0000","last_became_peered":"2024-04-26T21:48:02.367691+0000","last_unstale":"2024-04-26T21:48:03.777690+0000","last_undegraded":"2024-04-26T21:48:03.777690+0000","last_fullsized":"2024-04-26T21:48:03.777690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:20.518916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379117+0000","last_change":"2024-04-26T21:48:01.361790+0000","last_active":"2024-04-26T21:48:03.379117+0000","last_peered":"2024-04-26T21:48:03.379117+0000","last_clean":"2024-04-26T21:48:03.379117+0000","last_became_active":"2024-04-26T21:48:01.361442+0000","last_became_peered":"2024-04-26T21:48:01.361442+0000","last_unstale":"2024-04-26T21:48:03.379117+0000","last_undegraded":"2024-04-26T21:48:03.379117+0000","last_fullsized":"2024-04-26T21:48:03.379117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:53.685773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777642+0000","last_change":"2024-04-26T21:48:02.353275+0000","last_active":"2024-04-26T21:48:03.777642+0000","last_peered":"2024-04-26T21:48:03.777642+0000","last_clean":"2024-04-26T21:48:03.777642+0000","last_became_active":"2024-04-26T21:48:02.352986+0000","last_became_peered":"2024-04-26T21:48:02.352986+0000","last_unstale":"2024-04-26T21:48:03.777642+0000","last_undegraded":"2024-04-26T21:48:03.777642+0000","last_fullsized":"2024-04-26T21:48:03.777642+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:07.588078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361304+0000","last_change":"2024-04-26T21:48:01.358304+0000","last_active":"2024-04-26T21:48:03.361304+0000","last_peered":"2024-04-26T21:48:03.361304+0000","last_clean":"2024-04-26T21:48:03.361304+0000","last_became_active":"2024-04-26T21:48:01.357983+0000","last_became_peered":"2024-04-26T21:48:01.357983+0000","last_unstale":"2024-04-26T21:48:03.361304+0000","last_undegraded":"2024-04-26T21:48:03.361304+0000","last_fullsized":"2024-04-26T21:48:03.361304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:36.617628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090678+0000","last_change":"2024-04-26T21:48:02.356679+0000","last_active":"2024-04-26T21:48:04.090678+0000","last_peered":"2024-04-26T21:48:04.090678+0000","last_clean":"2024-04-26T21:48:04.090678+0000","last_became_active":"2024-04-26T21:48:02.355197+0000","last_became_peered":"2024-04-26T21:48:02.355197+0000","last_unstale":"2024-04-26T21:48:04.090678+0000","last_undegraded":"2024-04-26T21:48:04.090678+0000","last_fullsized":"2024-04-26T21:48:04.090678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:37.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777671+0000","last_change":"2024-04-26T21:48:01.350335+0000","last_active":"2024-04-26T21:48:03.777671+0000","last_peered":"2024-04-26T21:48:03.777671+0000","last_clean":"2024-04-26T21:48:03.777671+0000","last_became_active":"2024-04-26T21:48:01.349972+0000","last_became_peered":"2024-04-26T21:48:01.349972+0000","last_unstale":"2024-04-26T21:48:03.777671+0000","last_undegraded":"2024-04-26T21:48:03.777671+0000","last_fullsized":"2024-04-26T21:48:03.777671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:09.132827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379414+0000","last_change":"2024-04-26T21:48:02.356345+0000","last_active":"2024-04-26T21:48:03.379414+0000","last_peered":"2024-04-26T21:48:03.379414+0000","last_clean":"2024-04-26T21:48:03.379414+0000","last_became_active":"2024-04-26T21:48:02.355783+0000","last_became_peered":"2024-04-26T21:48:02.355783+0000","last_unstale":"2024-04-26T21:48:03.379414+0000","last_undegraded":"2024-04-26T21:48:03.379414+0000","last_fullsized":"2024-04-26T21:48:03.379414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:25.899452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379429+0000","last_change":"2024-04-26T21:48:01.354200+0000","last_active":"2024-04-26T21:48:03.379429+0000","last_peered":"2024-04-26T21:48:03.379429+0000","last_clean":"2024-04-26T21:48:03.379429+0000","last_became_active":"2024-04-26T21:48:01.353586+0000","last_became_peered":"2024-04-26T21:48:01.353586+0000","last_unstale":"2024-04-26T21:48:03.379429+0000","last_undegraded":"2024-04-26T21:48:03.379429+0000","last_fullsized":"2024-04-26T21:48:03.379429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:47.145203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360939+0000","last_change":"2024-04-26T21:48:02.349890+0000","last_active":"2024-04-26T21:48:03.360939+0000","last_peered":"2024-04-26T21:48:03.360939+0000","last_clean":"2024-04-26T21:48:03.360939+0000","last_became_active":"2024-04-26T21:48:02.349547+0000","last_became_peered":"2024-04-26T21:48:02.349547+0000","last_unstale":"2024-04-26T21:48:03.360939+0000","last_undegraded":"2024-04-26T21:48:03.360939+0000","last_fullsized":"2024-04-26T21:48:03.360939+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:42.674622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.779188+0000","last_change":"2024-04-26T21:48:01.348457+0000","last_active":"2024-04-26T21:48:03.779188+0000","last_peered":"2024-04-26T21:48:03.779188+0000","last_clean":"2024-04-26T21:48:03.779188+0000","last_became_active":"2024-04-26T21:48:01.348179+0000","last_became_peered":"2024-04-26T21:48:01.348179+0000","last_unstale":"2024-04-26T21:48:03.779188+0000","last_undegraded":"2024-04-26T21:48:03.779188+0000","last_fullsized":"2024-04-26T21:48:03.779188+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:06.950702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373124+0000","last_change":"2024-04-26T21:48:02.356203+0000","last_active":"2024-04-26T21:48:03.373124+0000","last_peered":"2024-04-26T21:48:03.373124+0000","last_clean":"2024-04-26T21:48:03.373124+0000","last_became_active":"2024-04-26T21:48:02.355913+0000","last_became_peered":"2024-04-26T21:48:02.355913+0000","last_unstale":"2024-04-26T21:48:03.373124+0000","last_undegraded":"2024-04-26T21:48:03.373124+0000","last_fullsized":"2024-04-26T21:48:03.373124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:32.860868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360273+0000","last_change":"2024-04-26T21:48:01.365810+0000","last_active":"2024-04-26T21:48:03.360273+0000","last_peered":"2024-04-26T21:48:03.360273+0000","last_clean":"2024-04-26T21:48:03.360273+0000","last_became_active":"2024-04-26T21:48:01.365524+0000","last_became_peered":"2024-04-26T21:48:01.365524+0000","last_unstale":"2024-04-26T21:48:03.360273+0000","last_undegraded":"2024-04-26T21:48:03.360273+0000","last_fullsized":"2024-04-26T21:48:03.360273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:12.917781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.359529+0000","last_change":"2024-04-26T21:48:02.354635+0000","last_active":"2024-04-26T21:48:03.359529+0000","last_peered":"2024-04-26T21:48:03.359529+0000","last_clean":"2024-04-26T21:48:03.359529+0000","last_became_active":"2024-04-26T21:48:02.354305+0000","last_became_peered":"2024-04-26T21:48:02.354305+0000","last_unstale":"2024-04-26T21:48:03.359529+0000","last_undegraded":"2024-04-26T21:48:03.359529+0000","last_fullsized":"2024-04-26T21:48:03.359529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:16.481924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361020+0000","last_change":"2024-04-26T21:48:01.349463+0000","last_active":"2024-04-26T21:48:03.361020+0000","last_peered":"2024-04-26T21:48:03.361020+0000","last_clean":"2024-04-26T21:48:03.361020+0000","last_became_active":"2024-04-26T21:48:01.349133+0000","last_became_peered":"2024-04-26T21:48:01.349133+0000","last_unstale":"2024-04-26T21:48:03.361020+0000","last_undegraded":"2024-04-26T21:48:03.361020+0000","last_fullsized":"2024-04-26T21:48:03.361020+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:36.592255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089852+0000","last_change":"2024-04-26T21:48:02.364680+0000","last_active":"2024-04-26T21:48:04.089852+0000","last_peered":"2024-04-26T21:48:04.089852+0000","last_clean":"2024-04-26T21:48:04.089852+0000","last_became_active":"2024-04-26T21:48:02.363821+0000","last_became_peered":"2024-04-26T21:48:02.363821+0000","last_unstale":"2024-04-26T21:48:04.089852+0000","last_undegraded":"2024-04-26T21:48:04.089852+0000","last_fullsized":"2024-04-26T21:48:04.089852+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:07.966352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373186+0000","last_change":"2024-04-26T21:48:01.354790+0000","last_active":"2024-04-26T21:48:03.373186+0000","last_peered":"2024-04-26T21:48:03.373186+0000","last_clean":"2024-04-26T21:48:03.373186+0000","last_became_active":"2024-04-26T21:48:01.354464+0000","last_became_peered":"2024-04-26T21:48:01.354464+0000","last_unstale":"2024-04-26T21:48:03.373186+0000","last_undegraded":"2024-04-26T21:48:03.373186+0000","last_fullsized":"2024-04-26T21:48:03.373186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:06.572429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778031+0000","last_change":"2024-04-26T21:48:02.355939+0000","last_active":"2024-04-26T21:48:03.778031+0000","last_peered":"2024-04-26T21:48:03.778031+0000","last_clean":"2024-04-26T21:48:03.778031+0000","last_became_active":"2024-04-26T21:48:02.355633+0000","last_became_peered":"2024-04-26T21:48:02.355633+0000","last_unstale":"2024-04-26T21:48:03.778031+0000","last_undegraded":"2024-04-26T21:48:03.778031+0000","last_fullsized":"2024-04-26T21:48:03.778031+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:21.682442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.359657+0000","last_change":"2024-04-26T21:48:01.351165+0000","last_active":"2024-04-26T21:48:03.359657+0000","last_peered":"2024-04-26T21:48:03.359657+0000","last_clean":"2024-04-26T21:48:03.359657+0000","last_became_active":"2024-04-26T21:48:01.350714+0000","last_became_peered":"2024-04-26T21:48:01.350714+0000","last_unstale":"2024-04-26T21:48:03.359657+0000","last_undegraded":"2024-04-26T21:48:03.359657+0000","last_fullsized":"2024-04-26T21:48:03.359657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:35.949642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372501+0000","last_change":"2024-04-26T21:48:02.356771+0000","last_active":"2024-04-26T21:48:03.372501+0000","last_peered":"2024-04-26T21:48:03.372501+0000","last_clean":"2024-04-26T21:48:03.372501+0000","last_became_active":"2024-04-26T21:48:02.356484+0000","last_became_peered":"2024-04-26T21:48:02.356484+0000","last_unstale":"2024-04-26T21:48:03.372501+0000","last_undegraded":"2024-04-26T21:48:03.372501+0000","last_fullsized":"2024-04-26T21:48:03.372501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:13.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372971+0000","last_change":"2024-04-26T21:48:01.353683+0000","last_active":"2024-04-26T21:48:03.372971+0000","last_peered":"2024-04-26T21:48:03.372971+0000","last_clean":"2024-04-26T21:48:03.372971+0000","last_became_active":"2024-04-26T21:48:01.353322+0000","last_became_peered":"2024-04-26T21:48:01.353322+0000","last_unstale":"2024-04-26T21:48:03.372971+0000","last_undegraded":"2024-04-26T21:48:03.372971+0000","last_fullsized":"2024-04-26T21:48:03.372971+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:53.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360115+0000","last_change":"2024-04-26T21:48:02.350035+0000","last_active":"2024-04-26T21:48:03.360115+0000","last_peered":"2024-04-26T21:48:03.360115+0000","last_clean":"2024-04-26T21:48:03.360115+0000","last_became_active":"2024-04-26T21:48:02.349718+0000","last_became_peered":"2024-04-26T21:48:02.349718+0000","last_unstale":"2024-04-26T21:48:03.360115+0000","last_undegraded":"2024-04-26T21:48:03.360115+0000","last_fullsized":"2024-04-26T21:48:03.360115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:58.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778017+0000","last_change":"2024-04-26T21:48:01.361093+0000","last_active":"2024-04-26T21:48:03.778017+0000","last_peered":"2024-04-26T21:48:03.778017+0000","last_clean":"2024-04-26T21:48:03.778017+0000","last_became_active":"2024-04-26T21:48:01.360155+0000","last_became_peered":"2024-04-26T21:48:01.360155+0000","last_unstale":"2024-04-26T21:48:03.778017+0000","last_undegraded":"2024-04-26T21:48:03.778017+0000","last_fullsized":"2024-04-26T21:48:03.778017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:53.685773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361054+0000","last_change":"2024-04-26T21:48:02.357095+0000","last_active":"2024-04-26T21:48:03.361054+0000","last_peered":"2024-04-26T21:48:03.361054+0000","last_clean":"2024-04-26T21:48:03.361054+0000","last_became_active":"2024-04-26T21:48:02.356784+0000","last_became_peered":"2024-04-26T21:48:02.356784+0000","last_unstale":"2024-04-26T21:48:03.361054+0000","last_undegraded":"2024-04-26T21:48:03.361054+0000","last_fullsized":"2024-04-26T21:48:03.361054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:51.890205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360818+0000","last_change":"2024-04-26T21:48:01.353765+0000","last_active":"2024-04-26T21:48:03.360818+0000","last_peered":"2024-04-26T21:48:03.360818+0000","last_clean":"2024-04-26T21:48:03.360818+0000","last_became_active":"2024-04-26T21:48:01.352051+0000","last_became_peered":"2024-04-26T21:48:01.352051+0000","last_unstale":"2024-04-26T21:48:03.360818+0000","last_undegraded":"2024-04-26T21:48:03.360818+0000","last_fullsized":"2024-04-26T21:48:03.360818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:59.619314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.779196+0000","last_change":"2024-04-26T21:48:02.356468+0000","last_active":"2024-04-26T21:48:03.779196+0000","last_peered":"2024-04-26T21:48:03.779196+0000","last_clean":"2024-04-26T21:48:03.779196+0000","last_became_active":"2024-04-26T21:48:02.349636+0000","last_became_peered":"2024-04-26T21:48:02.349636+0000","last_unstale":"2024-04-26T21:48:03.779196+0000","last_undegraded":"2024-04-26T21:48:03.779196+0000","last_fullsized":"2024-04-26T21:48:03.779196+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:00.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379574+0000","last_change":"2024-04-26T21:48:01.361936+0000","last_active":"2024-04-26T21:48:03.379574+0000","last_peered":"2024-04-26T21:48:03.379574+0000","last_clean":"2024-04-26T21:48:03.379574+0000","last_became_active":"2024-04-26T21:48:01.361627+0000","last_became_peered":"2024-04-26T21:48:01.361627+0000","last_unstale":"2024-04-26T21:48:03.379574+0000","last_undegraded":"2024-04-26T21:48:03.379574+0000","last_fullsized":"2024-04-26T21:48:03.379574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:06.950702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090578+0000","last_change":"2024-04-26T21:48:02.352195+0000","last_active":"2024-04-26T21:48:04.090578+0000","last_peered":"2024-04-26T21:48:04.090578+0000","last_clean":"2024-04-26T21:48:04.090578+0000","last_became_active":"2024-04-26T21:48:02.351873+0000","last_became_peered":"2024-04-26T21:48:02.351873+0000","last_unstale":"2024-04-26T21:48:04.090578+0000","last_undegraded":"2024-04-26T21:48:04.090578+0000","last_fullsized":"2024-04-26T21:48:04.090578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:04.495465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361652+0000","last_change":"2024-04-26T21:48:01.360053+0000","last_active":"2024-04-26T21:48:03.361652+0000","last_peered":"2024-04-26T21:48:03.361652+0000","last_clean":"2024-04-26T21:48:03.361652+0000","last_became_active":"2024-04-26T21:48:01.359747+0000","last_became_peered":"2024-04-26T21:48:01.359747+0000","last_unstale":"2024-04-26T21:48:03.361652+0000","last_undegraded":"2024-04-26T21:48:03.361652+0000","last_fullsized":"2024-04-26T21:48:03.361652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:35.949642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090303+0000","last_change":"2024-04-26T21:48:02.360608+0000","last_active":"2024-04-26T21:48:04.090303+0000","last_peered":"2024-04-26T21:48:04.090303+0000","last_clean":"2024-04-26T21:48:04.090303+0000","last_became_active":"2024-04-26T21:48:02.358409+0000","last_became_peered":"2024-04-26T21:48:02.358409+0000","last_unstale":"2024-04-26T21:48:04.090303+0000","last_undegraded":"2024-04-26T21:48:04.090303+0000","last_fullsized":"2024-04-26T21:48:04.090303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:47.845905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373190+0000","last_change":"2024-04-26T21:48:01.362479+0000","last_active":"2024-04-26T21:48:03.373190+0000","last_peered":"2024-04-26T21:48:03.373190+0000","last_clean":"2024-04-26T21:48:03.373190+0000","last_became_active":"2024-04-26T21:48:01.361845+0000","last_became_peered":"2024-04-26T21:48:01.361845+0000","last_unstale":"2024-04-26T21:48:03.373190+0000","last_undegraded":"2024-04-26T21:48:03.373190+0000","last_fullsized":"2024-04-26T21:48:03.373190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.883802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373006+0000","last_change":"2024-04-26T21:48:02.356159+0000","last_active":"2024-04-26T21:48:03.373006+0000","last_peered":"2024-04-26T21:48:03.373006+0000","last_clean":"2024-04-26T21:48:03.373006+0000","last_became_active":"2024-04-26T21:48:02.355477+0000","last_became_peered":"2024-04-26T21:48:02.355477+0000","last_unstale":"2024-04-26T21:48:03.373006+0000","last_undegraded":"2024-04-26T21:48:03.373006+0000","last_fullsized":"2024-04-26T21:48:03.373006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:59.242327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373441+0000","last_change":"2024-04-26T21:48:01.358096+0000","last_active":"2024-04-26T21:48:03.373441+0000","last_peered":"2024-04-26T21:48:03.373441+0000","last_clean":"2024-04-26T21:48:03.373441+0000","last_became_active":"2024-04-26T21:48:01.357724+0000","last_became_peered":"2024-04-26T21:48:01.357724+0000","last_unstale":"2024-04-26T21:48:03.373441+0000","last_undegraded":"2024-04-26T21:48:03.373441+0000","last_fullsized":"2024-04-26T21:48:03.373441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:19.503267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089301+0000","last_change":"2024-04-26T21:48:02.352167+0000","last_active":"2024-04-26T21:48:04.089301+0000","last_peered":"2024-04-26T21:48:04.089301+0000","last_clean":"2024-04-26T21:48:04.089301+0000","last_became_active":"2024-04-26T21:48:02.351202+0000","last_became_peered":"2024-04-26T21:48:02.351202+0000","last_unstale":"2024-04-26T21:48:04.089301+0000","last_undegraded":"2024-04-26T21:48:04.089301+0000","last_fullsized":"2024-04-26T21:48:04.089301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:37.638059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361500+0000","last_change":"2024-04-26T21:48:01.359987+0000","last_active":"2024-04-26T21:48:03.361500+0000","last_peered":"2024-04-26T21:48:03.361500+0000","last_clean":"2024-04-26T21:48:03.361500+0000","last_became_active":"2024-04-26T21:48:01.359253+0000","last_became_peered":"2024-04-26T21:48:01.359253+0000","last_unstale":"2024-04-26T21:48:03.361500+0000","last_undegraded":"2024-04-26T21:48:03.361500+0000","last_fullsized":"2024-04-26T21:48:03.361500+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:06.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361220+0000","last_change":"2024-04-26T21:48:02.350459+0000","last_active":"2024-04-26T21:48:03.361220+0000","last_peered":"2024-04-26T21:48:03.361220+0000","last_clean":"2024-04-26T21:48:03.361220+0000","last_became_active":"2024-04-26T21:48:02.350124+0000","last_became_peered":"2024-04-26T21:48:02.350124+0000","last_unstale":"2024-04-26T21:48:03.361220+0000","last_undegraded":"2024-04-26T21:48:03.361220+0000","last_fullsized":"2024-04-26T21:48:03.361220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:54.739888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777459+0000","last_change":"2024-04-26T21:48:01.360668+0000","last_active":"2024-04-26T21:48:03.777459+0000","last_peered":"2024-04-26T21:48:03.777459+0000","last_clean":"2024-04-26T21:48:03.777459+0000","last_became_active":"2024-04-26T21:48:01.360199+0000","last_became_peered":"2024-04-26T21:48:01.360199+0000","last_unstale":"2024-04-26T21:48:03.777459+0000","last_undegraded":"2024-04-26T21:48:03.777459+0000","last_fullsized":"2024-04-26T21:48:03.777459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:47.605190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777512+0000","last_change":"2024-04-26T21:48:02.358698+0000","last_active":"2024-04-26T21:48:03.777512+0000","last_peered":"2024-04-26T21:48:03.777512+0000","last_clean":"2024-04-26T21:48:03.777512+0000","last_became_active":"2024-04-26T21:48:02.358333+0000","last_became_peered":"2024-04-26T21:48:02.358333+0000","last_unstale":"2024-04-26T21:48:03.777512+0000","last_undegraded":"2024-04-26T21:48:03.777512+0000","last_fullsized":"2024-04-26T21:48:03.777512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:25.899452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.358947+0000","last_change":"2024-04-26T21:48:01.352934+0000","last_active":"2024-04-26T21:48:03.358947+0000","last_peered":"2024-04-26T21:48:03.358947+0000","last_clean":"2024-04-26T21:48:03.358947+0000","last_became_active":"2024-04-26T21:48:01.352618+0000","last_became_peered":"2024-04-26T21:48:01.352618+0000","last_unstale":"2024-04-26T21:48:03.358947+0000","last_undegraded":"2024-04-26T21:48:03.358947+0000","last_fullsized":"2024-04-26T21:48:03.358947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:36.622409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.090787+0000","last_change":"2024-04-26T21:48:02.364629+0000","last_active":"2024-04-26T21:48:04.090787+0000","last_peered":"2024-04-26T21:48:04.090787+0000","last_clean":"2024-04-26T21:48:04.090787+0000","last_became_active":"2024-04-26T21:48:02.364296+0000","last_became_peered":"2024-04-26T21:48:02.364296+0000","last_unstale":"2024-04-26T21:48:04.090787+0000","last_undegraded":"2024-04-26T21:48:04.090787+0000","last_fullsized":"2024-04-26T21:48:04.090787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:07.758660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.377173+0000","last_change":"2024-04-26T21:48:01.358679+0000","last_active":"2024-04-26T21:48:03.377173+0000","last_peered":"2024-04-26T21:48:03.377173+0000","last_clean":"2024-04-26T21:48:03.377173+0000","last_became_active":"2024-04-26T21:48:01.358429+0000","last_became_peered":"2024-04-26T21:48:01.358429+0000","last_unstale":"2024-04-26T21:48:03.377173+0000","last_undegraded":"2024-04-26T21:48:03.377173+0000","last_fullsized":"2024-04-26T21:48:03.377173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:59.334546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.778727+0000","last_change":"2024-04-26T21:48:02.354706+0000","last_active":"2024-04-26T21:48:03.778727+0000","last_peered":"2024-04-26T21:48:03.778727+0000","last_clean":"2024-04-26T21:48:03.778727+0000","last_became_active":"2024-04-26T21:48:02.354381+0000","last_became_peered":"2024-04-26T21:48:02.354381+0000","last_unstale":"2024-04-26T21:48:03.778727+0000","last_undegraded":"2024-04-26T21:48:03.778727+0000","last_fullsized":"2024-04-26T21:48:03.778727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:42.674622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360542+0000","last_change":"2024-04-26T21:48:01.364638+0000","last_active":"2024-04-26T21:48:03.360542+0000","last_peered":"2024-04-26T21:48:03.360542+0000","last_clean":"2024-04-26T21:48:03.360542+0000","last_became_active":"2024-04-26T21:48:01.363059+0000","last_became_peered":"2024-04-26T21:48:01.363059+0000","last_unstale":"2024-04-26T21:48:03.360542+0000","last_undegraded":"2024-04-26T21:48:03.360542+0000","last_fullsized":"2024-04-26T21:48:03.360542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:59.502886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.359983+0000","last_change":"2024-04-26T21:48:02.348488+0000","last_active":"2024-04-26T21:48:03.359983+0000","last_peered":"2024-04-26T21:48:03.359983+0000","last_clean":"2024-04-26T21:48:03.359983+0000","last_became_active":"2024-04-26T21:48:02.348212+0000","last_became_peered":"2024-04-26T21:48:02.348212+0000","last_unstale":"2024-04-26T21:48:03.359983+0000","last_undegraded":"2024-04-26T21:48:03.359983+0000","last_fullsized":"2024-04-26T21:48:03.359983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:13.951053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089669+0000","last_change":"2024-04-26T21:48:01.360831+0000","last_active":"2024-04-26T21:48:04.089669+0000","last_peered":"2024-04-26T21:48:04.089669+0000","last_clean":"2024-04-26T21:48:04.089669+0000","last_became_active":"2024-04-26T21:48:01.360531+0000","last_became_peered":"2024-04-26T21:48:01.360531+0000","last_unstale":"2024-04-26T21:48:04.089669+0000","last_undegraded":"2024-04-26T21:48:04.089669+0000","last_fullsized":"2024-04-26T21:48:04.089669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:46.830256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.373587+0000","last_change":"2024-04-26T21:48:02.363826+0000","last_active":"2024-04-26T21:48:03.373587+0000","last_peered":"2024-04-26T21:48:03.373587+0000","last_clean":"2024-04-26T21:48:03.373587+0000","last_became_active":"2024-04-26T21:48:02.363558+0000","last_became_peered":"2024-04-26T21:48:02.363558+0000","last_unstale":"2024-04-26T21:48:03.373587+0000","last_undegraded":"2024-04-26T21:48:03.373587+0000","last_fullsized":"2024-04-26T21:48:03.373587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:10.893317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.361029+0000","last_change":"2024-04-26T21:48:01.365725+0000","last_active":"2024-04-26T21:48:03.361029+0000","last_peered":"2024-04-26T21:48:03.361029+0000","last_clean":"2024-04-26T21:48:03.361029+0000","last_became_active":"2024-04-26T21:48:01.365438+0000","last_became_peered":"2024-04-26T21:48:01.365438+0000","last_unstale":"2024-04-26T21:48:03.361029+0000","last_undegraded":"2024-04-26T21:48:03.361029+0000","last_fullsized":"2024-04-26T21:48:03.361029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:36.617628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.372768+0000","last_change":"2024-04-26T21:48:02.364081+0000","last_active":"2024-04-26T21:48:03.372768+0000","last_peered":"2024-04-26T21:48:03.372768+0000","last_clean":"2024-04-26T21:48:03.372768+0000","last_became_active":"2024-04-26T21:48:02.361792+0000","last_became_peered":"2024-04-26T21:48:02.361792+0000","last_unstale":"2024-04-26T21:48:03.372768+0000","last_undegraded":"2024-04-26T21:48:03.372768+0000","last_fullsized":"2024-04-26T21:48:03.372768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:29.673155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.379244+0000","last_change":"2024-04-26T21:48:01.358639+0000","last_active":"2024-04-26T21:48:03.379244+0000","last_peered":"2024-04-26T21:48:03.379244+0000","last_clean":"2024-04-26T21:48:03.379244+0000","last_became_active":"2024-04-26T21:48:01.358358+0000","last_became_peered":"2024-04-26T21:48:01.358358+0000","last_unstale":"2024-04-26T21:48:03.379244+0000","last_undegraded":"2024-04-26T21:48:03.379244+0000","last_fullsized":"2024-04-26T21:48:03.379244+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:40.039550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.360698+0000","last_change":"2024-04-26T21:48:02.357798+0000","last_active":"2024-04-26T21:48:03.360698+0000","last_peered":"2024-04-26T21:48:03.360698+0000","last_clean":"2024-04-26T21:48:03.360698+0000","last_became_active":"2024-04-26T21:48:02.357376+0000","last_became_peered":"2024-04-26T21:48:02.357376+0000","last_unstale":"2024-04-26T21:48:03.360698+0000","last_undegraded":"2024-04-26T21:48:03.360698+0000","last_fullsized":"2024-04-26T21:48:03.360698+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:07.588078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:04.089505+0000","last_change":"2024-04-26T21:48:01.356300+0000","last_active":"2024-04-26T21:48:04.089505+0000","last_peered":"2024-04-26T21:48:04.089505+0000","last_clean":"2024-04-26T21:48:04.089505+0000","last_became_active":"2024-04-26T21:48:01.356022+0000","last_became_peered":"2024-04-26T21:48:01.356022+0000","last_unstale":"2024-04-26T21:48:04.089505+0000","last_undegraded":"2024-04-26T21:48:04.089505+0000","last_fullsized":"2024-04-26T21:48:04.089505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:43:47.145203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.378801+0000","last_change":"2024-04-26T21:48:02.354461+0000","last_active":"2024-04-26T21:48:03.378801+0000","last_peered":"2024-04-26T21:48:03.378801+0000","last_clean":"2024-04-26T21:48:03.378801+0000","last_became_active":"2024-04-26T21:48:02.351574+0000","last_became_peered":"2024-04-26T21:48:02.351574+0000","last_unstale":"2024-04-26T21:48:03.378801+0000","last_undegraded":"2024-04-26T21:48:03.378801+0000","last_fullsized":"2024-04-26T21:48:03.378801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:36.965291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.777937+0000","last_change":"2024-04-26T21:48:01.352343+0000","last_active":"2024-04-26T21:48:03.777937+0000","last_peered":"2024-04-26T21:48:03.777937+0000","last_clean":"2024-04-26T21:48:03.777937+0000","last_became_active":"2024-04-26T21:48:01.351231+0000","last_became_peered":"2024-04-26T21:48:01.351231+0000","last_unstale":"2024-04-26T21:48:03.777937+0000","last_undegraded":"2024-04-26T21:48:03.777937+0000","last_fullsized":"2024-04-26T21:48:03.777937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:40.039550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.779066+0000","last_change":"2024-04-26T21:48:02.361721+0000","last_active":"2024-04-26T21:48:03.779066+0000","last_peered":"2024-04-26T21:48:03.779066+0000","last_clean":"2024-04-26T21:48:03.779066+0000","last_became_active":"2024-04-26T21:48:02.361269+0000","last_became_peered":"2024-04-26T21:48:02.361269+0000","last_unstale":"2024-04-26T21:48:03.779066+0000","last_undegraded":"2024-04-26T21:48:03.779066+0000","last_fullsized":"2024-04-26T21:48:03.779066+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:01.307732+0000","last_clean_scrub_stamp":"2024-04-26T21:48:01.307732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:36.068313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.362335+0000","last_change":"2024-04-26T21:48:01.359917+0000","last_active":"2024-04-26T21:48:03.362335+0000","last_peered":"2024-04-26T21:48:03.362335+0000","last_clean":"2024-04-26T21:48:03.362335+0000","last_became_active":"2024-04-26T21:48:01.359158+0000","last_became_peered":"2024-04-26T21:48:01.359158+0000","last_unstale":"2024-04-26T21:48:03.362335+0000","last_undegraded":"2024-04-26T21:48:03.362335+0000","last_fullsized":"2024-04-26T21:48:03.362335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:48:00.292082+0000","last_clean_scrub_stamp":"2024-04-26T21:48:00.292082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:12.917781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:48:03.358736+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:48:03.358736+0000","last_peered":"2024-04-26T21:48:03.358736+0000","last_clean":"2024-04-26T21:48:03.358736+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:48:03.358736+0000","last_undegraded":"2024-04-26T21:48:03.358736+0000","last_fullsized":"2024-04-26T21:48:03.358736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":215753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":805517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:48:07.260 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:48:07.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:48:07.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.515+0000 7f821f600640 1 Processor -- start 2024-04-26T21:48:07.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.515+0000 7f821f600640 1 -- start start 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f821f600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821815d550 0x7f821806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f821f600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 0x7f8218072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f821f600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f821815da90 con 0x7f8218073790 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f821f600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f821815dd70 con 0x7f821815d550 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f821f600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f821815e050 con 0x7f82180b03a0 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f8217e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821815d550 0x7f821806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f8217e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821815d550 0x7f821806dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46642/0 (socket says 172.21.15.106:46642) 2024-04-26T21:48:07.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.517+0000 7f8217e00640 1 -- 172.21.15.106:0/936417768 learned_addr learned my addr 172.21.15.106:0/936417768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:07.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f8217e00640 1 -- 172.21.15.106:0/936417768 >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 msgr2=0x7f8218072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:07.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f821d600640 1 --2- 172.21.15.106:0/936417768 >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 0x7f8218072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f8217e00640 1 --2- 172.21.15.106:0/936417768 >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 0x7f8218072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f8217e00640 1 -- 172.21.15.106:0/936417768 >> v1:172.21.15.204:6789/0 conn(0x7f8218073790 legacy=0x7f8218073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f8217e00640 1 -- 172.21.15.106:0/936417768 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821815e330 con 0x7f821815d550 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.518+0000 7f821d600640 1 --2- 172.21.15.106:0/936417768 >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 0x7f8218072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.519+0000 7f8217e00640 1 --2- 172.21.15.106:0/936417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821815d550 0x7f821806dde0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82180d0c90 tx=0x7f820402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9756e79eb7a7b7be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.519+0000 7f8217400640 1 -- 172.21.15.106:0/936417768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204037070 con 0x7f821815d550 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.519+0000 7f8217400640 1 -- 172.21.15.106:0/936417768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8204032440 con 0x7f821815d550 2024-04-26T21:48:07.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.519+0000 7f8217400640 1 -- 172.21.15.106:0/936417768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204032810 con 0x7f821815d550 2024-04-26T21:48:07.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 -- 172.21.15.106:0/936417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821815d550 msgr2=0x7f821806dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 --2- 172.21.15.106:0/936417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821815d550 0x7f821806dde0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82180d0c90 tx=0x7f820402a230 comp rx=0 tx=0).stop 2024-04-26T21:48:07.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 -- 172.21.15.106:0/936417768 shutdown_connections 2024-04-26T21:48:07.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 --2- 172.21.15.106:0/936417768 >> v2:172.21.15.204:3300/0 conn(0x7f82180b03a0 0x7f8218072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 --2- 172.21.15.106:0/936417768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821815d550 0x7f821806dde0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 -- 172.21.15.106:0/936417768 >> 172.21.15.106:0/936417768 conn(0x7f82180b2a30 msgr2=0x7f82180b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:07.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 -- 172.21.15.106:0/936417768 shutdown_connections 2024-04-26T21:48:07.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.520+0000 7f821f600640 1 -- 172.21.15.106:0/936417768 wait complete. 2024-04-26T21:48:07.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.522+0000 7f821f600640 1 Processor -- start 2024-04-26T21:48:07.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.522+0000 7f821f600640 1 -- start start 2024-04-26T21:48:07.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82180b03a0 0x7f82181601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 0x7f8218158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82180ae020 con 0x7f8218073790 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82180adea0 con 0x7f821815d550 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8218072ba0 con 0x7f82180b03a0 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46656/0 (socket says 172.21.15.106:46656) 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.523+0000 7f821cc00640 1 -- 172.21.15.106:0/1166488536 learned_addr learned my addr 172.21.15.106:0/1166488536 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821d600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 0x7f8218158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f8217e00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82180b03a0 0x7f82181601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 -- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82180b03a0 msgr2=0x7f82181601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82180b03a0 0x7f82181601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 -- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 msgr2=0x7f8218158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 0x7f8218158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 -- 172.21.15.106:0/1166488536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8218158fc0 con 0x7f8218073790 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821d600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 0x7f8218158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:07.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821cc00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c410 tx=0x7f820c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c017070 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c0035f0 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0039e0 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.524+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181592a0 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.525+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8218159860 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.526+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8218069650 con 0x7f8218073790 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.527+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f820c01a070 con 0x7f8218073790 2024-04-26T21:48:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.528+0000 7f8214c00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 0x7f81f806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.528+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f820c08b2f0 con 0x7f8218073790 2024-04-26T21:48:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.528+0000 7f8217e00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 0x7f81f806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:07.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.529+0000 7f8217e00640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 0x7f81f806ef60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8204034000 tx=0x7f820402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:07.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.532+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f820c059530 con 0x7f8218073790 2024-04-26T21:48:07.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.809+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f82180b27a0 con 0x7f8218073790 2024-04-26T21:48:07.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.814+0000 7f8214c00640 1 -- 172.21.15.106:0/1166488536 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f820c058e30 con 0x7f8218073790 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBRICxmLaFgExAAICat8SKNtzXa+6k1HQv30Q== 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:48:07.815 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 msgr2=0x7f81f806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 0x7f81f806ef60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8204034000 tx=0x7f820402e000 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 msgr2=0x7f821815a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c410 tx=0x7f820c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 shutdown_connections 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821815d550 0x7f8218158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81f806caa0 0x7f81f806ef60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82180b03a0 0x7f82181601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 --2- 172.21.15.106:0/1166488536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8218073790 0x7f821815a060 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:07.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 >> 172.21.15.106:0/1166488536 conn(0x7f82180b2a30 msgr2=0x7f821806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:07.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 shutdown_connections 2024-04-26T21:48:07.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:07.817+0000 7f821f600640 1 -- 172.21.15.106:0/1166488536 wait complete. 2024-04-26T21:48:07.837 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.028+0000 7f5811000640 1 Processor -- start 2024-04-26T21:48:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.028+0000 7f5811000640 1 -- start start 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5811000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f580c0d5d90 0x7f580c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5811000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580c0b5d30 con 0x7f580c15ef60 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5811000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580c0b6010 con 0x7f580c0d34a0 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5811000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f580c0b62f0 con 0x7f580c0d5d90 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5809e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f580c15ef60 0x7f580c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39906/0 (socket says 172.21.15.106:39906) 2024-04-26T21:48:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5809e00640 1 -- 172.21.15.106:0/495043907 learned_addr learned my addr 172.21.15.106:0/495043907 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.030+0000 7f5809400640 1 --2- 172.21.15.106:0/495043907 >> v2:172.21.15.204:3300/0 conn(0x7f580c0d5d90 0x7f580c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078175257 0 0) 0x7f580c0b5d30 con 0x7f580c15ef60 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57f0003520 con 0x7f580c15ef60 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747386091 0 0) 0x7f57f0003520 con 0x7f580c15ef60 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 >> v2:172.21.15.204:3300/0 conn(0x7f580c0d5d90 msgr2=0x7f580c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 --2- 172.21.15.106:0/495043907 >> v2:172.21.15.204:3300/0 conn(0x7f580c0d5d90 0x7f580c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 >> v1:172.21.15.204:6789/0 conn(0x7f580c0d34a0 legacy=0x7f580c0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c0b65d0 con 0x7f580c15ef60 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f57f4001670 con 0x7f580c15ef60 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 >> v1:172.21.15.106:6789/0 conn(0x7f580c15ef60 legacy=0x7f580c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f57f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57f0004800 0x7f57f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57f00051c0 0x7f57f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f580c0b62f0 con 0x7f57f0003a00 2024-04-26T21:48:08.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000c0a0 con 0x7f57f0004800 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57f000c2a0 con 0x7f57f00051c0 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.031+0000 7f5809e00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f57f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809400640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57f0004800 0x7f57f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809e00640 1 -- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57f00051c0 msgr2=0x7f57f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809e00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57f00051c0 0x7f57f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809e00640 1 -- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57f0004800 msgr2=0x7f57f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809e00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57f0004800 0x7f57f0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f5809e00640 1 -- 172.21.15.106:0/495043907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f0003520 con 0x7f57f0003a00 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.032+0000 7f580a800640 1 -- 172.21.15.106:0/495043907 reap_dead start 2024-04-26T21:48:08.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.033+0000 7f5809e00640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f57f0003e20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c0e0 tx=0x7f57f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0851e81a51b622 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.033+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4007c20 con 0x7f57f0003a00 2024-04-26T21:48:08.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.033+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4005af0 con 0x7f57f0003a00 2024-04-26T21:48:08.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.033+0000 7f5808a00640 1 -- 172.21.15.106:0/495043907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4007070 con 0x7f57f0003a00 2024-04-26T21:48:08.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 -- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 msgr2=0x7f57f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f57f0003e20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c0e0 tx=0x7f57f0005830 comp rx=0 tx=0).stop 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 -- 172.21.15.106:0/495043907 shutdown_connections 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 --2- 172.21.15.106:0/495043907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f57f0003e20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 -- 172.21.15.106:0/495043907 >> 172.21.15.106:0/495043907 conn(0x7f580c09e3b0 msgr2=0x7f580c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 -- 172.21.15.106:0/495043907 shutdown_connections 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.034+0000 7f5811000640 1 -- 172.21.15.106:0/495043907 wait complete. 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.036+0000 7f5811000640 1 Processor -- start 2024-04-26T21:48:08.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.036+0000 7f5811000640 1 -- start start 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f580c151ed0 0x7f580c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 0x7f580c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f580c15f4c0 con 0x7f57f0003a00 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f580c15f340 con 0x7f580c06db60 2024-04-26T21:48:08.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5811000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f580c15f640 con 0x7f580c151ed0 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5809e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5809e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57238/0 (socket says 172.21.15.106:57238) 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5809e00640 1 -- 172.21.15.106:0/3398622878 learned_addr learned my addr 172.21.15.106:0/3398622878 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f580a800640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 0x7f580c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5809400640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f580c151ed0 0x7f580c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.037+0000 7f5809e00640 1 -- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f580c151ed0 msgr2=0x7f580c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5809e00640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f580c151ed0 0x7f580c15f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5809e00640 1 -- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 msgr2=0x7f580c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5809e00640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 0x7f580c06dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5809e00640 1 -- 172.21.15.106:0/3398622878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c15fd90 con 0x7f57f0003a00 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f580a800640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 0x7f580c06dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5809e00640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c0e0 tx=0x7f580c0b61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4041020 con 0x7f57f0003a00 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4038040 con 0x7f57f0003a00 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4001670 con 0x7f57f0003a00 2024-04-26T21:48:08.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c161570 con 0x7f57f0003a00 2024-04-26T21:48:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.038+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c161b30 con 0x7f57f0003a00 2024-04-26T21:48:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.040+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580c0b3d70 con 0x7f57f0003a00 2024-04-26T21:48:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.041+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f57f4044020 con 0x7f57f0003a00 2024-04-26T21:48:08.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.041+0000 7f5802000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 0x7f57e806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.042+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f57f40ac260 con 0x7f57f0003a00 2024-04-26T21:48:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.042+0000 7f5809400640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 0x7f57e806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.042+0000 7f5809400640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 0x7f57e806ef60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cea90 tx=0x7f57fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.046+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f580c0b3d70 con 0x7f57f0003a00 2024-04-26T21:48:08.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.284+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f580c0d4330 con 0x7f57f0003a00 2024-04-26T21:48:08.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.285+0000 7f5802000640 1 -- 172.21.15.106:0/3398622878 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5190 (secure 0 0 0) 0x7f580c0d4330 con 0x7f57f0003a00 2024-04-26T21:48:08.286 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:08.286 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:48:08.286 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 25 2024-04-26T21:48:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 msgr2=0x7f57e806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 0x7f57e806ef60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cea90 tx=0x7f57fc008040 comp rx=0 tx=0).stop 2024-04-26T21:48:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 msgr2=0x7f580c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c0e0 tx=0x7f580c0b61e0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 shutdown_connections 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f580c06db60 0x7f580c06dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f57e806caa0 0x7f57e806ef60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f580c151ed0 0x7f580c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 --2- 172.21.15.106:0/3398622878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f0003a00 0x7f580c0cb620 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 >> 172.21.15.106:0/3398622878 conn(0x7f580c09e3b0 msgr2=0x7f580c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 shutdown_connections 2024-04-26T21:48:08.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.288+0000 7f5811000640 1 -- 172.21.15.106:0/3398622878 wait complete. 2024-04-26T21:48:08.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:48:01.789843+0000', 'modified': '2024-04-26T21:48:04.848684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5778': {'gid': 5778, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/1457746115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 1457746115}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1457746115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5778, 'qdb_cluster': [5778]} 2024-04-26T21:48:08.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:48:08.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:48:08.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.921+0000 7f2797200640 1 Processor -- start 2024-04-26T21:48:08.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.922+0000 7f2797200640 1 -- start start 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.923+0000 7f2797200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27980d03b0 0x7f2798166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.923+0000 7f2797200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27980b5ff0 con 0x7f27981644e0 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.923+0000 7f2797200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2798167d40 con 0x7f27980d1580 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.923+0000 7f2797200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2798167a60 con 0x7f27980d03b0 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.923+0000 7f278fe00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f27981644e0 0x7f27980d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39914/0 (socket says 172.21.15.106:39914) 2024-04-26T21:48:08.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f278fe00640 1 -- 172.21.15.106:0/76455544 learned_addr learned my addr 172.21.15.106:0/76455544 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795e00640 1 --2- 172.21.15.106:0/76455544 >> v2:172.21.15.204:3300/0 conn(0x7f27980d03b0 0x7f2798166ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700173195 0 0) 0x7f27980b5ff0 con 0x7f27981644e0 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f277c003520 con 0x7f27981644e0 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3103718369 0 0) 0x7f277c003520 con 0x7f27981644e0 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 >> v2:172.21.15.204:3300/0 conn(0x7f27980d03b0 msgr2=0x7f2798166ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 --2- 172.21.15.106:0/76455544 >> v2:172.21.15.204:3300/0 conn(0x7f27980d03b0 0x7f2798166ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 >> v1:172.21.15.204:6789/0 conn(0x7f27980d1580 legacy=0x7f2798072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2798163a00 con 0x7f27981644e0 2024-04-26T21:48:08.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2780001670 con 0x7f27981644e0 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 >> v1:172.21.15.106:6789/0 conn(0x7f27981644e0 legacy=0x7f27980d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 0x7f277c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.924+0000 7f2795400640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c004800 0x7f277c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795400640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f277c0052e0 0x7f277c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2798167a60 con 0x7f277c0039b0 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f277c00c1b0 con 0x7f277c004800 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f277c00c3b0 con 0x7f277c0052e0 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 0x7f277c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 -- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f277c0052e0 msgr2=0x7f277c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f277c0052e0 0x7f277c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 -- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c004800 msgr2=0x7f277c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c004800 0x7f277c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2795e00640 1 -- 172.21.15.106:0/76455544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c003520 con 0x7f277c0039b0 2024-04-26T21:48:08.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.925+0000 7f2796800640 1 -- 172.21.15.106:0/76455544 reap_dead start 2024-04-26T21:48:08.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.926+0000 7f2795e00640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 0x7f277c003dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f278800ba00 tx=0x7f277c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a883b8e81dbd19e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.926+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788030070 con 0x7f277c0039b0 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278802ad80 con 0x7f277c0039b0 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2795400640 1 -- 172.21.15.106:0/76455544 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278800b600 con 0x7f277c0039b0 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 -- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 msgr2=0x7f277c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 0x7f277c003dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f278800ba00 tx=0x7f277c005830 comp rx=0 tx=0).stop 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 -- 172.21.15.106:0/76455544 shutdown_connections 2024-04-26T21:48:08.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 --2- 172.21.15.106:0/76455544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c0039b0 0x7f277c003dd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 -- 172.21.15.106:0/76455544 >> 172.21.15.106:0/76455544 conn(0x7f279806c740 msgr2=0x7f279806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 -- 172.21.15.106:0/76455544 shutdown_connections 2024-04-26T21:48:08.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.927+0000 7f2797200640 1 -- 172.21.15.106:0/76455544 wait complete. 2024-04-26T21:48:08.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.929+0000 7f2797200640 1 Processor -- start 2024-04-26T21:48:08.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.929+0000 7f2797200640 1 -- start start 2024-04-26T21:48:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 0x7f27981ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2798156b20 0x7f2798156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f27980d0790 con 0x7f27981f9600 2024-04-26T21:48:08.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2798163fc0 con 0x7f277c0039b0 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f2797200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27980d0910 con 0x7f2798156b20 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f278fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f278fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57264/0 (socket says 172.21.15.106:57264) 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.930+0000 7f278fe00640 1 -- 172.21.15.106:0/921178815 learned_addr learned my addr 172.21.15.106:0/921178815 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f2795e00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 0x7f27981ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f2796800640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2798156b20 0x7f2798156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 -- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2798156b20 msgr2=0x7f2798156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2798156b20 0x7f2798156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 -- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 msgr2=0x7f27981ee700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 0x7f27981ee700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 -- 172.21.15.106:0/921178815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981f9b50 con 0x7f27981f9600 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f2795e00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 0x7f27981ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:08.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278fe00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f278000c700 tx=0x7f278000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278e000640 1 -- 172.21.15.106:0/921178815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780018070 con 0x7f27981f9600 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278e000640 1 -- 172.21.15.106:0/921178815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27800072f0 con 0x7f27981f9600 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f278e000640 1 -- 172.21.15.106:0/921178815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800047e0 con 0x7f27981f9600 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.931+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981f9e30 con 0x7f27981f9600 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.932+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981fa3f0 con 0x7f27981f9600 2024-04-26T21:48:08.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.935+0000 7f278e000640 1 -- 172.21.15.106:0/921178815 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f278001b070 con 0x7f27981f9600 2024-04-26T21:48:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.936+0000 7f278e000640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 0x7f277406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.936+0000 7f278e000640 1 -- 172.21.15.106:0/921178815 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f278008b3d0 con 0x7f27981f9600 2024-04-26T21:48:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.936+0000 7f2795e00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 0x7f277406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.936+0000 7f279fd94740 1 Processor -- start 2024-04-26T21:48:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.936+0000 7f279fd94740 1 -- start start 2024-04-26T21:48:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f2795e00640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 0x7f277406ef90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277c004780 tx=0x7f2788030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a10910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a0eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eef8972600 con 0x55eef8a39270 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eef890f100 con 0x55eef8a4ae80 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55eef896e5c0 con 0x55eef8a0e7b0 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57280/0 (socket says 172.21.15.106:57280) 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.937+0000 7f2796800640 1 -- 172.21.15.106:0/1655420616 learned_addr learned my addr 172.21.15.106:0/1655420616 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2795e00640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a10910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f278fe00640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a0eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 -- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 msgr2=0x55eef8a10910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a10910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 -- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 msgr2=0x55eef8a0eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a0eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 -- 172.21.15.106:0/1655420616 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef86842d0 con 0x55eef8a39270 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2795e00640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a10910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f278fe00640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a0eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2796800640 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f279806f660 tx=0x7f2790002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896e3816a0921284 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2786a00640 1 -- 172.21.15.106:0/1655420616 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790003970 con 0x55eef8a39270 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2786a00640 1 -- 172.21.15.106:0/1655420616 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2790003b30 con 0x55eef8a39270 2024-04-26T21:48:08.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.938+0000 7f2786a00640 1 -- 172.21.15.106:0/1655420616 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000d790 con 0x55eef8a39270 2024-04-26T21:48:08.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 -- 172.21.15.106:0/1655420616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 msgr2=0x55eef8a396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f279806f660 tx=0x7f2790002090 comp rx=0 tx=0).stop 2024-04-26T21:48:08.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 -- 172.21.15.106:0/1655420616 shutdown_connections 2024-04-26T21:48:08.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a396f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a0eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 --2- 172.21.15.106:0/1655420616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a10910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 -- 172.21.15.106:0/1655420616 >> 172.21.15.106:0/1655420616 conn(0x55eef89a2d10 msgr2=0x55eef8a54270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 -- 172.21.15.106:0/1655420616 shutdown_connections 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.939+0000 7f279fd94740 1 -- 172.21.15.106:0/1655420616 wait complete. 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.940+0000 7f279fd94740 1 -- start start 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.940+0000 7f279fd94740 1 Processor -- start 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a34650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a55db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eef88f9f70 con 0x55eef8a4ae80 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eef8a36dc0 con 0x55eef8a39270 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55eef8a642d0 con 0x55eef8a0e7b0 2024-04-26T21:48:08.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.941+0000 7f279fd94740 10 client.0 _ll_register_callbacks cb 0x55eef8968480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57294/0 (socket says 172.21.15.106:57294) 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 -- 172.21.15.106:0/4068716121 learned_addr learned my addr 172.21.15.106:0/4068716121 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2795e00640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a34650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f278fe00640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a55db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 -- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 msgr2=0x55eef8a34650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a34650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 -- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 msgr2=0x55eef8a55db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a55db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 -- 172.21.15.106:0/4068716121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef874e6b0 con 0x55eef8a4ae80 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f2796800640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2790012000 tx=0x7f27900014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9570a6a3d805e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.942+0000 7f277b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:48:08.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790023020 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f279000dc20 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2790003b30 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f279002d050 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 10 client.5913 fetch_fsmap learned FSMap version 25 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55eef8a562f0 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2790003b30 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f2790030020 con 0x55eef8a4ae80 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 10 client.5913 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 10 client.5913 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 20 client.5913 populate_metadata read hostname 'smithi106' 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a3b750 0x55eef8a3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 4 client.5913 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.943+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 --> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef8980210 con 0x55eef8a3b750 2024-04-26T21:48:08.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.944+0000 7f2795e00640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a3b750 0x55eef8a3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.944+0000 7f2795e00640 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a3b750 0x55eef8a3bb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9369b7cbb5d586e server_cookie=d5e76d5741dccdb2 in_seq=0 out_seq=0 2024-04-26T21:48:08.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.944+0000 7f277b400640 10 client.5913 ms_handle_connect on v2:172.21.15.106:6836/1457746115 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.945+0000 7f277b400640 1 -- 172.21.15.106:0/4068716121 <== mds.0 v2:172.21.15.106:6836/1457746115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eef8980210 con 0x55eef8a3b750 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.945+0000 7f277b400640 10 client.5913 handle_command_reply: tid=0 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.945+0000 7f279fd94740 1 client.5913 shutdown 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.945+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 msgr2=0x55eef8a56c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.945+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2790012000 tx=0x7f27900014a0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 shutdown_connections 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a56c30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a3b750 0x55eef8a3bb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a55db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 --2- 172.21.15.106:0/4068716121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a34650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 >> 172.21.15.106:0/4068716121 conn(0x55eef89a2d10 msgr2=0x55eef8a36fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f277b400640 0 client.5913 ms_handle_reset on v2:172.21.15.106:6836/1457746115 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 shutdown_connections 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 1 -- 172.21.15.106:0/4068716121 wait complete. 2024-04-26T21:48:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.946+0000 7f279fd94740 20 client.5913 trim_cache size 0 max 16384 2024-04-26T21:48:08.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.966+0000 7f279fd94740 1 Processor -- start 2024-04-26T21:48:08.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.966+0000 7f279fd94740 1 -- start start 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a33aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2798205280 con 0x55eef8a4ae80 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2798145be0 con 0x55eef8a39270 2024-04-26T21:48:08.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2798146230 con 0x55eef8a0e7b0 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57302/0 (socket says 172.21.15.106:57302) 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 -- 172.21.15.106:0/3185766696 learned_addr learned my addr 172.21.15.106:0/3185766696 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 -- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 msgr2=0x55eef8a33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 -- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 msgr2=0x55eef8a33aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a33aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.967+0000 7f2796800640 1 -- 172.21.15.106:0/3185766696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef8a04370 con 0x55eef8a4ae80 2024-04-26T21:48:08.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f2796800640 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2790009010 tx=0x7f279001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb8f11c84d7a8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f2786000640 1 -- 172.21.15.106:0/3185766696 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790023040 con 0x55eef8a4ae80 2024-04-26T21:48:08.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f2786000640 1 -- 172.21.15.106:0/3185766696 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279000da20 con 0x55eef8a4ae80 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f2786000640 1 -- 172.21.15.106:0/3185766696 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000de10 con 0x55eef8a4ae80 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 -- 172.21.15.106:0/3185766696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 msgr2=0x55eef8a4d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2790009010 tx=0x7f279001e000 comp rx=0 tx=0).stop 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 -- 172.21.15.106:0/3185766696 shutdown_connections 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a4ae80 0x55eef8a4d2b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a39270 0x55eef8a33aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 --2- 172.21.15.106:0/3185766696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.968+0000 7f279fd94740 1 -- 172.21.15.106:0/3185766696 >> 172.21.15.106:0/3185766696 conn(0x55eef89a2d10 msgr2=0x55eef8a04df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.969+0000 7f279fd94740 1 -- 172.21.15.106:0/3185766696 shutdown_connections 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.969+0000 7f279fd94740 1 -- 172.21.15.106:0/3185766696 wait complete. 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.969+0000 7f279fd94740 1 -- start start 2024-04-26T21:48:08.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.969+0000 7f279fd94740 1 Processor -- start 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a49c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eef8a63ba0 con 0x55eef8a39270 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eef8a64010 con 0x55eef8a4ae80 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55eef8a3a620 con 0x55eef8a0e7b0 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.970+0000 7f279fd94740 10 client.0 _ll_register_callbacks cb 0x7f2790001ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57312/0 (socket says 172.21.15.106:57312) 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 -- 172.21.15.106:0/1079047203 learned_addr learned my addr 172.21.15.106:0/1079047203 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 -- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 msgr2=0x55eef8a37b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f2796800640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a49c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 -- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 msgr2=0x55eef8a49c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a49c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 -- 172.21.15.106:0/1079047203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef8a3bcd0 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f278fe00640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2780089660 tx=0x7f27800595b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63723063adc69583 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f277b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.971+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780089a60 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2780071180 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279000da20 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2780061a60 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 10 client.5919 fetch_fsmap learned FSMap version 25 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55eef8a56e60 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279000da20 con 0x55eef8a39270 2024-04-26T21:48:08.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f2780058240 con 0x55eef8a39270 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 10 client.5919 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 10 client.5919 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 20 client.5919 populate_metadata read hostname 'smithi106' 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a36280 0x55eef8a36660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 4 client.5919 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 --> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] -- command(tid 0: {"prefix": "status"}) -- 0x55eef8a38d60 con 0x55eef8a36280 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.972+0000 7f2795e00640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a36280 0x55eef8a36660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f2795e00640 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a36280 0x55eef8a36660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd3cf360243ee73 server_cookie=8d65dd227f9cf00e in_seq=0 out_seq=0 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f277b400640 10 client.5919 ms_handle_connect on v2:172.21.15.106:6836/1457746115 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f277b400640 1 -- 172.21.15.106:0/1079047203 <== mds.0 v2:172.21.15.106:6836/1457746115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55eef8a38d60 con 0x55eef8a36280 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f277b400640 10 client.5919 handle_command_reply: tid=0 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 client.5919 shutdown 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 msgr2=0x55eef8a380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2780089660 tx=0x7f27800595b0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 shutdown_connections 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:6836/1457746115,v1:172.21.15.106:6837/1457746115] conn(0x55eef8a36280 0x55eef8a36660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eef8a4ae80 0x55eef8a49c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef8a39270 0x55eef8a380b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 --2- 172.21.15.106:0/1079047203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55eef8a0e7b0 0x55eef8a37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f277b400640 0 client.5919 ms_handle_reset on v2:172.21.15.106:6836/1457746115 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.973+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 >> 172.21.15.106:0/1079047203 conn(0x55eef89a2d10 msgr2=0x55eef8a33560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 shutdown_connections 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f279fd94740 1 -- 172.21.15.106:0/1079047203 wait complete. 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f279fd94740 20 client.5919 trim_cache size 0 max 16384 2024-04-26T21:48:08.977 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e6ee84ed-cbe5-4ff1-aa0d-97a291446523", 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5778, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 6.6336013219999996, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 25, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 45, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 45, 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 13.126010346999999 2024-04-26T21:48:08.978 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:48:08.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 msgr2=0x7f277406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 0x7f277406ef90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277c004780 tx=0x7f2788030000 comp rx=0 tx=0).stop 2024-04-26T21:48:08.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 msgr2=0x7f2798166080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:08.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f278000c700 tx=0x7f278000cbc0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 shutdown_connections 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.974+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f277406cad0 0x7f277406ef90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2798156b20 0x7f2798156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27981f9600 0x7f2798166080 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 --2- 172.21.15.106:0/921178815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f277c0039b0 0x7f27981ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 >> 172.21.15.106:0/921178815 conn(0x7f279806c740 msgr2=0x7f2798071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 shutdown_connections 2024-04-26T21:48:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:08.975+0000 7f2797200640 1 -- 172.21.15.106:0/921178815 wait complete. 2024-04-26T21:48:08.992 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:48:09.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.193+0000 7f78ba400640 1 Processor -- start 2024-04-26T21:48:09.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.194+0000 7f78ba400640 1 -- start start 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.195+0000 7f78ba400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78b415cdf0 0x7f78b40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.195+0000 7f78ba400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f78b40cd650 0x7f78b406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.195+0000 7f78ba400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b40af9a0 con 0x7f78b40b4f50 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.195+0000 7f78ba400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f78b40ae2c0 con 0x7f78b415cdf0 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.195+0000 7f78ba400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f78b40b05b0 con 0x7f78b40cd650 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78b415cdf0 0x7f78b40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78b415cdf0 0x7f78b40add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57326/0 (socket says 172.21.15.106:57326) 2024-04-26T21:48:09.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 -- 172.21.15.106:0/4216959713 learned_addr learned my addr 172.21.15.106:0/4216959713 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78abe00640 1 --2- 172.21.15.106:0/4216959713 >> v2:172.21.15.204:3300/0 conn(0x7f78b40cd650 0x7f78b406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 -- 172.21.15.106:0/4216959713 >> v2:172.21.15.204:3300/0 conn(0x7f78b40cd650 msgr2=0x7f78b406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 --2- 172.21.15.106:0/4216959713 >> v2:172.21.15.204:3300/0 conn(0x7f78b40cd650 0x7f78b406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 -- 172.21.15.106:0/4216959713 >> v1:172.21.15.204:6789/0 conn(0x7f78b40b4f50 legacy=0x7f78b4072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.196+0000 7f78b3400640 1 -- 172.21.15.106:0/4216959713 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b40afc80 con 0x7f78b415cdf0 2024-04-26T21:48:09.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.197+0000 7f78b3400640 1 --2- 172.21.15.106:0/4216959713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b415cdf0 0x7f78b40add80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f78b40d1d00 tx=0x7f78a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb46a6eb2a3134 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:09.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.197+0000 7f78b2a00640 1 -- 172.21.15.106:0/4216959713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4037070 con 0x7f78b415cdf0 2024-04-26T21:48:09.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.197+0000 7f78b2a00640 1 -- 172.21.15.106:0/4216959713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a4032440 con 0x7f78b415cdf0 2024-04-26T21:48:09.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.197+0000 7f78b2a00640 1 -- 172.21.15.106:0/4216959713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4032810 con 0x7f78b415cdf0 2024-04-26T21:48:09.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 -- 172.21.15.106:0/4216959713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b415cdf0 msgr2=0x7f78b40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 --2- 172.21.15.106:0/4216959713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b415cdf0 0x7f78b40add80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f78b40d1d00 tx=0x7f78a402a230 comp rx=0 tx=0).stop 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 -- 172.21.15.106:0/4216959713 shutdown_connections 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 --2- 172.21.15.106:0/4216959713 >> v2:172.21.15.204:3300/0 conn(0x7f78b40cd650 0x7f78b406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 --2- 172.21.15.106:0/4216959713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b415cdf0 0x7f78b40add80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 -- 172.21.15.106:0/4216959713 >> 172.21.15.106:0/4216959713 conn(0x7f78b40b2d50 msgr2=0x7f78b40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 -- 172.21.15.106:0/4216959713 shutdown_connections 2024-04-26T21:48:09.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.198+0000 7f78ba400640 1 -- 172.21.15.106:0/4216959713 wait complete. 2024-04-26T21:48:09.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.200+0000 7f78ba400640 1 Processor -- start 2024-04-26T21:48:09.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.200+0000 7f78ba400640 1 -- start start 2024-04-26T21:48:09.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 0x7f78b409caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78b415cdf0 0x7f78b4148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78b415d1d0 con 0x7f78b40cd650 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78b40b0240 con 0x7f78b40b4f50 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.201+0000 7f78ba400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78b415d350 con 0x7f78b415cdf0 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57330/0 (socket says 172.21.15.106:57330) 2024-04-26T21:48:09.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 -- 172.21.15.106:0/672551738 learned_addr learned my addr 172.21.15.106:0/672551738 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:09.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78b3400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 0x7f78b409caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78b3e00640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78b415cdf0 0x7f78b4148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 -- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78b415cdf0 msgr2=0x7f78b4148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78b415cdf0 0x7f78b4148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 -- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 msgr2=0x7f78b409caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 0x7f78b409caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 -- 172.21.15.106:0/672551738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b4146260 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78b3400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 0x7f78b409caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.202+0000 7f78abe00640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f789c00d800 tx=0x7f789c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.203+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c00b6c0 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.203+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c019070 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.203+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c00ba80 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.203+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b409d560 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.203+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b409db20 con 0x7f78b40cd650 2024-04-26T21:48:09.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.204+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b4069540 con 0x7f78b40cd650 2024-04-26T21:48:09.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.205+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f789c00bc20 con 0x7f78b40cd650 2024-04-26T21:48:09.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.206+0000 7f78ab400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 0x7f789006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:09.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.206+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f789c08c3e0 con 0x7f78b40cd650 2024-04-26T21:48:09.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.206+0000 7f78b3400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 0x7f789006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:09.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.207+0000 7f78b3400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 0x7f789006ef60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f78a4034000 tx=0x7f78a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:09.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.210+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f789c01c360 con 0x7f78b40cd650 2024-04-26T21:48:09.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.453+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78b40b56d0 con 0x7f78b40cd650 2024-04-26T21:48:09.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.454+0000 7f78ab400640 1 -- 172.21.15.106:0/672551738 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f789c05a4f0 con 0x7f78b40cd650 2024-04-26T21:48:09.454 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:48:09.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 msgr2=0x7f789006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 0x7f789006ef60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f78a4034000 tx=0x7f78a402e000 comp rx=0 tx=0).stop 2024-04-26T21:48:09.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 msgr2=0x7f78b41485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f789c00d800 tx=0x7f789c00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 shutdown_connections 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f789006caa0 0x7f789006ef60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78b415cdf0 0x7f78b4148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78b40cd650 0x7f78b41485f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 --2- 172.21.15.106:0/672551738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b40b4f50 0x7f78b409caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 >> 172.21.15.106:0/672551738 conn(0x7f78b40b2d50 msgr2=0x7f78b40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 shutdown_connections 2024-04-26T21:48:09.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09.457+0000 7f78ba400640 1 -- 172.21.15.106:0/672551738 wait complete. 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:48:09.477 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:48:09.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:09.477 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-26T21:48:09.528 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42469sec preferred_lft 42469sec 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::c49c:8bff:fe17:2a52/64 scope link 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 4a:0c:21:55:3a:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:09.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::480c:21ff:fe55:3af6/64 scope link 2024-04-26T21:48:09.530 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.530 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:48:09.530 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 62:45:ae:c9:5b:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:48:09.530 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::6045:aeff:fec9:5b20/64 scope link 2024-04-26T21:48:09.530 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:09.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:09.531 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:48:09.603 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-26T21:48:09.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:09.606 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-26T21:48:09.652 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-26T21:48:09.652 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:48:09.652 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:48:09.652 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:48:09.652 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:48:09.653 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-26T21:48:09.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:48:09.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:48:09.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:09.912 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:48:09.957 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:48:09.957 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:48:09.958 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:48:09.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:09.958 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-26T21:48:10.040 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T21:48:10.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:48:10.041 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:10.093 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:48:10.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:48:10.093 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:10.144 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:48:10.228 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:48:10.285 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:48:10.286 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:48:10.287 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:48:10.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:10.288 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:48:10.337 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:48:10.337 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T21:48:10.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-26T21:48:10.412 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:48:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-26T21:48:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-26T21:48:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-26T21:48:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-26T21:48:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:48:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:48:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:48:10.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:10.454 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-26T21:48:10.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-26T21:48:10.513+0000 7fc21bdea480 -1 init, newargv = 0x55ff2fe0ed30 newargc=15 2024-04-26T21:48:10.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[50725]: starting ceph client 2024-04-26T21:48:10.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[50725]: starting fuse 2024-04-26T21:48:10.539 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-26T21:48:10.539 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-26T21:48:10.540 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-26T21:48:10.541 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-26T21:48:10.724 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.50725.asok 2024-04-26T21:48:10.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:48:10.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50725.asok 2024-04-26T21:48:10.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:10.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50725.asok status 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:48:10.959 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "50725", 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5931, 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5931 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 3776459629 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:48:10.960 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 3776459629 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.5931 192.168.0.1:0/3776459629", 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/3776459629", 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 25, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 45, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-26T21:48:10.961 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:48:10.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:10.975 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-26T21:48:10.975 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:11.027 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-26T21:48:11.027 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:11.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:11.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:11.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:48:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.315+0000 7f9f93600640 1 Processor -- start 2024-04-26T21:48:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.315+0000 7f9f93600640 1 -- start start 2024-04-26T21:48:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.316+0000 7f9f93600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c06c770 0x7f9f8c06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.316+0000 7f9f93600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8c15cfd0 0x7f9f8c073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.316+0000 7f9f93600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c06d090 con 0x7f9f8c15cb70 2024-04-26T21:48:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.316+0000 7f9f93600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c0af460 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.316+0000 7f9f93600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c0ae850 con 0x7f9f8c06c770 2024-04-26T21:48:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8c15cfd0 0x7f9f8c073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f8be00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9f8c15cb70 0x7f9f8c073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39926/0 (socket says 172.21.15.106:39926) 2024-04-26T21:48:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8c15cfd0 0x7f9f8c073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57338/0 (socket says 172.21.15.106:57338) 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f8be00640 1 -- 172.21.15.106:0/2954637882 learned_addr learned my addr 172.21.15.106:0/2954637882 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f90c00640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c06c770 0x7f9f8c06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 -- 172.21.15.106:0/2954637882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c06c770 msgr2=0x7f9f8c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c06c770 0x7f9f8c06cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 -- 172.21.15.106:0/2954637882 >> v1:172.21.15.106:6789/0 conn(0x7f9f8c15cb70 legacy=0x7f9f8c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f91600640 1 -- 172.21.15.106:0/2954637882 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0aeb30 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.317+0000 7f9f8b400640 1 -- 172.21.15.106:0/2954637882 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343883244 0 0) 0x7f9f8c06d090 con 0x7f9f8c15cb70 2024-04-26T21:48:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.319+0000 7f9f91600640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cfd0 0x7f9f8c073830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009d90 tx=0x7f9f8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17be4c513e3a2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.319+0000 7f9f8b400640 1 -- 172.21.15.106:0/2954637882 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007740 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.319+0000 7f9f8b400640 1 -- 172.21.15.106:0/2954637882 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f840078e0 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.319+0000 7f9f8b400640 1 -- 172.21.15.106:0/2954637882 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007cb0 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 -- 172.21.15.106:0/2954637882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cfd0 msgr2=0x7f9f8c073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cfd0 0x7f9f8c073830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009d90 tx=0x7f9f8402a660 comp rx=0 tx=0).stop 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 -- 172.21.15.106:0/2954637882 shutdown_connections 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cfd0 0x7f9f8c073830 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 --2- 172.21.15.106:0/2954637882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c06c770 0x7f9f8c06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 -- 172.21.15.106:0/2954637882 >> 172.21.15.106:0/2954637882 conn(0x7f9f8c0b2d50 msgr2=0x7f9f8c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 -- 172.21.15.106:0/2954637882 shutdown_connections 2024-04-26T21:48:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.320+0000 7f9f93600640 1 -- 172.21.15.106:0/2954637882 wait complete. 2024-04-26T21:48:11.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.321+0000 7f9f93600640 1 Processor -- start 2024-04-26T21:48:11.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.322+0000 7f9f93600640 1 -- start start 2024-04-26T21:48:11.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.322+0000 7f9f93600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f8c06c770 0x7f9f8c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f93600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f93600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c15cfd0 0x7f9f8c14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f93600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15c600 con 0x7f9f8c15cb70 2024-04-26T21:48:11.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f93600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0af0f0 con 0x7f9f8c06c770 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f93600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c15c780 con 0x7f9f8c15cfd0 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57352/0 (socket says 172.21.15.106:57352) 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 -- 172.21.15.106:0/1093119244 learned_addr learned my addr 172.21.15.106:0/1093119244 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 -- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c15cfd0 msgr2=0x7f9f8c14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f91600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c15cfd0 0x7f9f8c14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c15cfd0 0x7f9f8c14b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 -- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f8c06c770 msgr2=0x7f9f8c14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f8c06c770 0x7f9f8c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.323+0000 7f9f8be00640 1 -- 172.21.15.106:0/1093119244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0ca6a0 con 0x7f9f8c15cb70 2024-04-26T21:48:11.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f8be00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c4d0 tx=0x7f9f7400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74018070 con 0x7f9f8c15cb70 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74007500 con 0x7f9f8c15cb70 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004a20 con 0x7f9f8c15cb70 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c0732e0 con 0x7f9f8c15cb70 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.324+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c073b20 con 0x7f9f8c15cb70 2024-04-26T21:48:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.325+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c069540 con 0x7f9f8c15cb70 2024-04-26T21:48:11.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.326+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9f7401b070 con 0x7f9f8c15cb70 2024-04-26T21:48:11.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.327+0000 7f9f88c00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 0x7f9f6806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:11.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.327+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9f7408b1a0 con 0x7f9f8c15cb70 2024-04-26T21:48:11.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.327+0000 7f9f90c00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 0x7f9f6806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:11.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.328+0000 7f9f90c00640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 0x7f9f6806ef60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098f0 tx=0x7f9f7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:11.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.332+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f8c0732e0 con 0x7f9f8c15cb70 2024-04-26T21:48:11.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.575+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f8c06ec40 con 0x7f9f8c15cb70 2024-04-26T21:48:11.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.576+0000 7f9f88c00640 1 -- 172.21.15.106:0/1093119244 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f9f74059340 con 0x7f9f8c15cb70 2024-04-26T21:48:11.576 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 msgr2=0x7f9f6806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 0x7f9f6806ef60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098f0 tx=0x7f9f7c008040 comp rx=0 tx=0).stop 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 msgr2=0x7f9f8c15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c4d0 tx=0x7f9f7400c990 comp rx=0 tx=0).stop 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 shutdown_connections 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9f6806caa0 0x7f9f6806ef60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f8c15cfd0 0x7f9f8c14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c15cb70 0x7f9f8c15a6b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 --2- 172.21.15.106:0/1093119244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f8c06c770 0x7f9f8c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:11.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 >> 172.21.15.106:0/1093119244 conn(0x7f9f8c0b2d50 msgr2=0x7f9f8c0b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:11.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 shutdown_connections 2024-04-26T21:48:11.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:11.578+0000 7f9f93600640 1 -- 172.21.15.106:0/1093119244 wait complete. 2024-04-26T21:48:11.596 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:48:11.597 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:48:11.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:11.597 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:11.657 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42487sec preferred_lft 42487sec 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:3a:e4:9f:22:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::2818:afff:fe5c:ca4c/64 scope link 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.658 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ce:52:1d:14:cf:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::cc52:1dff:fe14:cf6d/64 scope link 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:3a:e4:9f:22:6a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::6c3a:e4ff:fe9f:226a/64 scope link 2024-04-26T21:48:11.659 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:48:11.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:11.660 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:48:11.737 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-26T21:48:11.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:11.742 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-04-26T21:48:11.793 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-04-26T21:48:11.793 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:48:11.793 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:48:11.793 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:48:11.793 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:48:11.794 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:48:11.794 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-04-26T21:48:11.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:48:11.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:48:11.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:48:11.795 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-04-26T21:48:11.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:48:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:48:12.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:48:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:48:12.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:12.012 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:48:12.058 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:48:12.059 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:48:12.060 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:48:12.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:12.060 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-04-26T21:48:12.132 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T21:48:12.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:48:12.132 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:12.182 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:48:12.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:48:12.183 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:12.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:48:12.311 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:48:12.365 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:48:12.366 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:48:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:48:12.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:12.368 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:48:12.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:48:12.422 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T21:48:12.427 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-04-26T21:48:12.492 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:48:12.533 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/selinux 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-04-26T21:48:12.534 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T21:48:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-04-26T21:48:12.536 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:48:12.536 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:48:12.536 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:48:12.536 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:48:12.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:12.537 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-04-26T21:48:12.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-04-26T21:48:12.593+0000 7f68acf1a480 -1 init, newargv = 0x5612da686390 newargc=15 2024-04-26T21:48:12.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[41148]: starting ceph client 2024-04-26T21:48:12.606 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[41148]: starting fuse 2024-04-26T21:48:12.622 INFO:teuthology.orchestra.run.smithi204.stdout:41 2024-04-26T21:48:12.622 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T21:48:12.622 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:48:12.622 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-04-26T21:48:12.622 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-04-26T21:48:12.622 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:48:12.623 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-04-26T21:48:12.624 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-04-26T21:48:12.813 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.41148.asok 2024-04-26T21:48:12.818 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:48:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:48:12.830 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41148.asok 2024-04-26T21:48:12.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:12.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41148.asok status 2024-04-26T21:48:13.051 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "41148", 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-04-26T21:48:13.052 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 5074, 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 5074 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 4152430564 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-04-26T21:48:13.053 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 4152430564 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.5074 192.168.0.1:0/4152430564", 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/4152430564", 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 25, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 45, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-04-26T21:48:13.054 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:48:13.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:13.066 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-04-26T21:48:13.066 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:13.121 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-04-26T21:48:13.121 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:13.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:13.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:13.206 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T21:48:13.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.418+0000 7f5436600640 1 Processor -- start 2024-04-26T21:48:13.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.418+0000 7f5436600640 1 -- start start 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f5436600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f5436600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f543006e210 0x7f5430072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f5436600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300b6010 con 0x7f543015f170 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f5436600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f54300726e0 con 0x7f543006e210 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f5436600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f543006e6a0 con 0x7f54301634b0 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f542fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f543006e210 0x7f5430072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f542f400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f543015f170 0x7f5430162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39930/0 (socket says 172.21.15.106:39930) 2024-04-26T21:48:13.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f542f400640 1 -- 172.21.15.106:0/3487579095 learned_addr learned my addr 172.21.15.106:0/3487579095 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.420+0000 7f542ea00640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.421+0000 7f542fe00640 1 -- 172.21.15.106:0/3487579095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 msgr2=0x7f54300b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.421+0000 7f542fe00640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.421+0000 7f542fe00640 1 -- 172.21.15.106:0/3487579095 >> v1:172.21.15.106:6789/0 conn(0x7f543015f170 legacy=0x7f5430162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.421+0000 7f542e000640 1 -- 172.21.15.106:0/3487579095 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751985754 0 0) 0x7f54300b6010 con 0x7f543015f170 2024-04-26T21:48:13.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.421+0000 7f542fe00640 1 -- 172.21.15.106:0/3487579095 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300b5d30 con 0x7f543006e210 2024-04-26T21:48:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.422+0000 7f542fe00640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430072b70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5428009d90 tx=0x7f542802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc13c49afdf8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.422+0000 7f542e000640 1 -- 172.21.15.106:0/3487579095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428007740 con 0x7f543006e210 2024-04-26T21:48:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.422+0000 7f542e000640 1 -- 172.21.15.106:0/3487579095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54280078e0 con 0x7f543006e210 2024-04-26T21:48:13.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.422+0000 7f542e000640 1 -- 172.21.15.106:0/3487579095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428007cb0 con 0x7f543006e210 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 -- 172.21.15.106:0/3487579095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 msgr2=0x7f5430072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430072b70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5428009d90 tx=0x7f542802a660 comp rx=0 tx=0).stop 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 -- 172.21.15.106:0/3487579095 shutdown_connections 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430072b70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 --2- 172.21.15.106:0/3487579095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 -- 172.21.15.106:0/3487579095 >> 172.21.15.106:0/3487579095 conn(0x7f54300d0c80 msgr2=0x7f54300d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:13.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 -- 172.21.15.106:0/3487579095 shutdown_connections 2024-04-26T21:48:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.423+0000 7f5436600640 1 -- 172.21.15.106:0/3487579095 wait complete. 2024-04-26T21:48:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.425+0000 7f5436600640 1 Processor -- start 2024-04-26T21:48:13.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.425+0000 7f5436600640 1 -- start start 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f543015f170 0x7f543009e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f543015f550 con 0x7f543006e210 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5430072370 con 0x7f543015f170 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f5436600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f54300730b0 con 0x7f54301634b0 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57364/0 (socket says 172.21.15.106:57364) 2024-04-26T21:48:13.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 -- 172.21.15.106:0/2677754812 learned_addr learned my addr 172.21.15.106:0/2677754812 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542fe00640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 -- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 msgr2=0x7f54300add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542ea00640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f543015f170 0x7f543009e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 -- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f543015f170 msgr2=0x7f543009e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f543015f170 0x7f543009e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.426+0000 7f542f400640 1 -- 172.21.15.106:0/2677754812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5430162130 con 0x7f543006e210 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f542f400640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f542000c4d0 tx=0x7f542000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420018070 con 0x7f543006e210 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5420007500 con 0x7f543006e210 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420004a20 con 0x7f543006e210 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5430162410 con 0x7f543006e210 2024-04-26T21:48:13.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.427+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543009e910 con 0x7f543006e210 2024-04-26T21:48:13.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.429+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5430069540 con 0x7f543006e210 2024-04-26T21:48:13.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.430+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f542001b070 con 0x7f543006e210 2024-04-26T21:48:13.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.430+0000 7f5427400640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 0x7f541006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:13.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.431+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f542008b270 con 0x7f543006e210 2024-04-26T21:48:13.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.431+0000 7f542ea00640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 0x7f541006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:13.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.431+0000 7f542ea00640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 0x7f541006ef60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54301f84f0 tx=0x7f541c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:13.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.435+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5420059380 con 0x7f543006e210 2024-04-26T21:48:13.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:13.668+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f543006def0 con 0x7f543006e210 2024-04-26T21:48:14.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.412+0000 7f5427400640 1 -- 172.21.15.106:0/2677754812 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) ==== 136+0+0 (secure 0 0 0) 0x7f5420058c80 con 0x7f543006e210 2024-04-26T21:48:14.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 msgr2=0x7f541006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 0x7f541006ef60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54301f84f0 tx=0x7f541c008040 comp rx=0 tx=0).stop 2024-04-26T21:48:14.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 msgr2=0x7f5430148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f542000c4d0 tx=0x7f542000c990 comp rx=0 tx=0).stop 2024-04-26T21:48:14.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 shutdown_connections 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f541006caa0 0x7f541006ef60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54301634b0 0x7f54300add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f543015f170 0x7f543009e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 --2- 172.21.15.106:0/2677754812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f543006e210 0x7f5430148810 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.417+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 >> 172.21.15.106:0/2677754812 conn(0x7f54300d0c80 msgr2=0x7f54300d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.418+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 shutdown_connections 2024-04-26T21:48:14.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.418+0000 7f5436600640 1 -- 172.21.15.106:0/2677754812 wait complete. 2024-04-26T21:48:14.439 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T21:48:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.630+0000 7f280b200640 1 Processor -- start 2024-04-26T21:48:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.630+0000 7f280b200640 1 -- start start 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.631+0000 7f280b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f280c0d0e70 0x7f280c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.631+0000 7f280b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 0x7f280c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.631+0000 7f280b200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f280c0b5d40 0x7f280c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280b200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f280c0d1790 con 0x7f280c0b5d40 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f280c0727f0 con 0x7f280c0d0e70 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0b4a10 con 0x7f280c0b5660 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f280c0b5d40 0x7f280c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f280c0b5d40 0x7f280c0aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57374/0 (socket says 172.21.15.106:57374) 2024-04-26T21:48:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 -- 172.21.15.106:0/2671181108 learned_addr learned my addr 172.21.15.106:0/2671181108 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:14.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f2801400640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 0x7f280c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:14.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 -- 172.21.15.106:0/2671181108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 msgr2=0x7f280c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 0x7f280c0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 -- 172.21.15.106:0/2671181108 >> v2:172.21.15.204:3300/0 conn(0x7f280c0d0e70 msgr2=0x7f280c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 --2- 172.21.15.106:0/2671181108 >> v2:172.21.15.204:3300/0 conn(0x7f280c0d0e70 0x7f280c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f280a800640 1 -- 172.21.15.106:0/2671181108 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0b4cf0 con 0x7f280c0b5d40 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.632+0000 7f2801400640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 0x7f280c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.633+0000 7f280a800640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5d40 0x7f280c0aee00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f280c0aff60 tx=0x7f280402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefabbc262918e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f2809400640 1 -- 172.21.15.106:0/2671181108 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804037070 con 0x7f280c0b5d40 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f2809400640 1 -- 172.21.15.106:0/2671181108 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804032440 con 0x7f280c0b5d40 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f2809400640 1 -- 172.21.15.106:0/2671181108 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804032810 con 0x7f280c0b5d40 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 -- 172.21.15.106:0/2671181108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5d40 msgr2=0x7f280c0aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5d40 0x7f280c0aee00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f280c0aff60 tx=0x7f280402a140 comp rx=0 tx=0).stop 2024-04-26T21:48:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 -- 172.21.15.106:0/2671181108 shutdown_connections 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5d40 0x7f280c0aee00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 --2- 172.21.15.106:0/2671181108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5660 0x7f280c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 --2- 172.21.15.106:0/2671181108 >> v2:172.21.15.204:3300/0 conn(0x7f280c0d0e70 0x7f280c0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 -- 172.21.15.106:0/2671181108 >> 172.21.15.106:0/2671181108 conn(0x7f280c06c740 msgr2=0x7f280c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.634+0000 7f280b200640 1 -- 172.21.15.106:0/2671181108 shutdown_connections 2024-04-26T21:48:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.635+0000 7f280b200640 1 -- 172.21.15.106:0/2671181108 wait complete. 2024-04-26T21:48:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.636+0000 7f280b200640 1 Processor -- start 2024-04-26T21:48:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.636+0000 7f280b200640 1 -- start start 2024-04-26T21:48:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5d40 0x7f280c145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f280c0d0e70 0x7f280c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b05b0 con 0x7f280c0b5660 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b0430 con 0x7f280c0d0e70 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f280b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f280c15f1a0 con 0x7f280c0b5d40 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.637+0000 7f2809e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57390/0 (socket says 172.21.15.106:57390) 2024-04-26T21:48:14.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 -- 172.21.15.106:0/3044156889 learned_addr learned my addr 172.21.15.106:0/3044156889 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2801400640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5d40 0x7f280c145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 -- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5d40 msgr2=0x7f280c145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5d40 0x7f280c145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 -- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f280c0d0e70 msgr2=0x7f280c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f280c0d0e70 0x7f280c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 -- 172.21.15.106:0/3044156889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c145a40 con 0x7f280c0b5660 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2809e00640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00ba80 tx=0x7f27fc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.638+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc019070 con 0x7f280c0b5660 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.639+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc00cc20 con 0x7f280c0b5660 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.639+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc014440 con 0x7f280c0b5660 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.639+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c155580 con 0x7f280c0b5660 2024-04-26T21:48:14.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.639+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c155840 con 0x7f280c0b5660 2024-04-26T21:48:14.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.640+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c0693c0 con 0x7f280c0b5660 2024-04-26T21:48:14.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.641+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f27fc01c070 con 0x7f280c0b5660 2024-04-26T21:48:14.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.642+0000 7f2802a00640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 0x7f27e806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:14.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.642+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f27fc08baf0 con 0x7f280c0b5660 2024-04-26T21:48:14.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.642+0000 7f2801400640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 0x7f27e806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:14.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.643+0000 7f2801400640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 0x7f27e806ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f280c1554f0 tx=0x7f27f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:14.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.646+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f280c145a40 con 0x7f280c0b5660 2024-04-26T21:48:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.863+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f280c0afd40 con 0x7f27e806caa0 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.866+0000 7f2802a00640 1 -- 172.21.15.106:0/3044156889 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f280c0afd40 con 0x7f27e806caa0 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-26T21:48:14.866 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.868+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 msgr2=0x7f27e806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.868+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 0x7f27e806ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f280c1554f0 tx=0x7f27f4008040 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.868+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 msgr2=0x7f280c152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.868+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00ba80 tx=0x7f27fc00bf40 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 shutdown_connections 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f280c0d0e70 0x7f280c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f27e806caa0 0x7f27e806ef60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c0b5d40 0x7f280c145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 --2- 172.21.15.106:0/3044156889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f280c0b5660 0x7f280c152a80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:14.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 >> 172.21.15.106:0/3044156889 conn(0x7f280c06c740 msgr2=0x7f280c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:14.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 shutdown_connections 2024-04-26T21:48:14.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:14.869+0000 7f280b200640 1 -- 172.21.15.106:0/3044156889 wait complete. 2024-04-26T21:48:14.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:48:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.091+0000 7fa197200640 1 Processor -- start 2024-04-26T21:48:15.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.091+0000 7fa197200640 1 -- start start 2024-04-26T21:48:15.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa197200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa190160d60 0x7fa1900b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa197200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa197200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1900b4d20 con 0x7fa190146280 2024-04-26T21:48:15.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa197200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900b5000 con 0x7fa190160d60 2024-04-26T21:48:15.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa197200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900b52e0 con 0x7fa190161140 2024-04-26T21:48:15.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.093+0000 7fa195400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:58162/0 (socket says 172.21.15.106:58162) 2024-04-26T21:48:15.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 -- 172.21.15.106:0/3802837262 learned_addr learned my addr 172.21.15.106:0/3802837262 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:15.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa196800640 1 --1- 172.21.15.106:0/3802837262 >> v1:172.21.15.204:6789/0 conn(0x7fa190146280 0x7fa1900cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:59228/0 (socket says 172.21.15.106:59228) 2024-04-26T21:48:15.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195e00640 1 --2- 172.21.15.106:0/3802837262 >> v2:172.21.15.204:3300/0 conn(0x7fa190160d60 0x7fa1900b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 -- 172.21.15.106:0/3802837262 >> v2:172.21.15.204:3300/0 conn(0x7fa190160d60 msgr2=0x7fa1900b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:15.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 --2- 172.21.15.106:0/3802837262 >> v2:172.21.15.204:3300/0 conn(0x7fa190160d60 0x7fa1900b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 -- 172.21.15.106:0/3802837262 >> v1:172.21.15.204:6789/0 conn(0x7fa190146280 legacy=0x7fa1900cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.094+0000 7fa195400640 1 -- 172.21.15.106:0/3802837262 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1900cf410 con 0x7fa190161140 2024-04-26T21:48:15.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.095+0000 7fa195400640 1 --2- 172.21.15.106:0/3802837262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa180009870 tx=0x7fa18002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fec79265c5a6d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.096+0000 7fa194a00640 1 -- 172.21.15.106:0/3802837262 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180037070 con 0x7fa190161140 2024-04-26T21:48:15.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.096+0000 7fa194a00640 1 -- 172.21.15.106:0/3802837262 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa180032440 con 0x7fa190161140 2024-04-26T21:48:15.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.096+0000 7fa194a00640 1 -- 172.21.15.106:0/3802837262 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180032810 con 0x7fa190161140 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 -- 172.21.15.106:0/3802837262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 msgr2=0x7fa19006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 --2- 172.21.15.106:0/3802837262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa180009870 tx=0x7fa18002a250 comp rx=0 tx=0).stop 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 -- 172.21.15.106:0/3802837262 shutdown_connections 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 --2- 172.21.15.106:0/3802837262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190161140 0x7fa19006dee0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 --2- 172.21.15.106:0/3802837262 >> v2:172.21.15.204:3300/0 conn(0x7fa190160d60 0x7fa1900b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 -- 172.21.15.106:0/3802837262 >> 172.21.15.106:0/3802837262 conn(0x7fa1900b2900 msgr2=0x7fa1900b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:15.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 -- 172.21.15.106:0/3802837262 shutdown_connections 2024-04-26T21:48:15.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.097+0000 7fa197200640 1 -- 172.21.15.106:0/3802837262 wait complete. 2024-04-26T21:48:15.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.099+0000 7fa197200640 1 Processor -- start 2024-04-26T21:48:15.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.099+0000 7fa197200640 1 -- start start 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190146280 0x7fa190159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 0x7fa190073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa19006e420 con 0x7fa190160d60 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900b6070 con 0x7fa190161140 2024-04-26T21:48:15.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa197200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa19006e5a0 con 0x7fa190146280 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa195400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa195400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57400/0 (socket says 172.21.15.106:57400) 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.100+0000 7fa195400640 1 -- 172.21.15.106:0/3358547200 learned_addr learned my addr 172.21.15.106:0/3358547200 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195e00640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190146280 0x7fa190159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa196800640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 0x7fa190073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 -- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190146280 msgr2=0x7fa190159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190146280 0x7fa190159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 -- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 msgr2=0x7fa190073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 0x7fa190073410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 -- 172.21.15.106:0/3358547200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1900cc380 con 0x7fa190160d60 2024-04-26T21:48:15.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa196800640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 0x7fa190073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa195400640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa1800090b0 tx=0x7fa1800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180043070 con 0x7fa190160d60 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa180038040 con 0x7fa190160d60 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180037050 con 0x7fa190160d60 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1900cc660 con 0x7fa190160d60 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.101+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1900ccc20 con 0x7fa190160d60 2024-04-26T21:48:15.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.103+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1900693c0 con 0x7fa190160d60 2024-04-26T21:48:15.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.104+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa180046070 con 0x7fa190160d60 2024-04-26T21:48:15.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.105+0000 7fa18a000640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 0x7fa17406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.105+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa1800abe30 con 0x7fa190160d60 2024-04-26T21:48:15.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.105+0000 7fa195e00640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 0x7fa17406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.105+0000 7fa195e00640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 0x7fa17406ef60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa184009a10 tx=0x7fa184008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.109+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa18007a070 con 0x7fa190160d60 2024-04-26T21:48:15.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.298+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa1900b3160 con 0x7fa190160d60 2024-04-26T21:48:15.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.299+0000 7fa18a000640 1 -- 172.21.15.106:0/3358547200 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fa180079970 con 0x7fa190160d60 2024-04-26T21:48:15.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 msgr2=0x7fa17406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 0x7fa17406ef60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa184009a10 tx=0x7fa184008040 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 msgr2=0x7fa190072ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa1800090b0 tx=0x7fa1800046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 shutdown_connections 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa190161140 0x7fa190073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190160d60 0x7fa190072ed0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa17406caa0 0x7fa17406ef60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 --2- 172.21.15.106:0/3358547200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa190146280 0x7fa190159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.301+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 >> 172.21.15.106:0/3358547200 conn(0x7fa1900b2900 msgr2=0x7fa1900cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.302+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 shutdown_connections 2024-04-26T21:48:15.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.302+0000 7fa197200640 1 -- 172.21.15.106:0/3358547200 wait complete. 2024-04-26T21:48:15.320 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:48:15.320 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:15.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:15.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.608+0000 7ff614600640 1 Processor -- start 2024-04-26T21:48:15.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.608+0000 7ff614600640 1 -- start start 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff614600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff60c15cb70 0x7ff60c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff614600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff60c15cfd0 0x7ff60c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff614600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c0ae850 con 0x7ff60c06c770 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff614600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff60c0af460 con 0x7ff60c15cfd0 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff614600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff60c0aeb30 con 0x7ff60c15cb70 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff612600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff60c15cfd0 0x7ff60c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff612600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff60c15cfd0 0x7ff60c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57416/0 (socket says 172.21.15.106:57416) 2024-04-26T21:48:15.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.610+0000 7ff612600640 1 -- 172.21.15.106:0/513408968 learned_addr learned my addr 172.21.15.106:0/513408968 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.611+0000 7ff612600640 1 -- 172.21.15.106:0/513408968 >> v2:172.21.15.204:3300/0 conn(0x7ff60c15cb70 msgr2=0x7ff60c06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.611+0000 7ff612600640 1 --2- 172.21.15.106:0/513408968 >> v2:172.21.15.204:3300/0 conn(0x7ff60c15cb70 0x7ff60c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.611+0000 7ff612600640 1 -- 172.21.15.106:0/513408968 >> v1:172.21.15.204:6789/0 conn(0x7ff60c06c770 legacy=0x7ff60c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:15.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.611+0000 7ff612600640 1 -- 172.21.15.106:0/513408968 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c0aee10 con 0x7ff60c15cfd0 2024-04-26T21:48:15.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.612+0000 7ff612600640 1 --2- 172.21.15.106:0/513408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f3a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0afec0 tx=0x7ff60402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd44479f6666c60f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.612+0000 7ff60be00640 1 -- 172.21.15.106:0/513408968 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604036070 con 0x7ff60c15cfd0 2024-04-26T21:48:15.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.612+0000 7ff60be00640 1 -- 172.21.15.106:0/513408968 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff604031600 con 0x7ff60c15cfd0 2024-04-26T21:48:15.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.612+0000 7ff60be00640 1 -- 172.21.15.106:0/513408968 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040319d0 con 0x7ff60c15cfd0 2024-04-26T21:48:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 -- 172.21.15.106:0/513408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 msgr2=0x7ff60c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 --2- 172.21.15.106:0/513408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f3a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0afec0 tx=0x7ff60402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 -- 172.21.15.106:0/513408968 shutdown_connections 2024-04-26T21:48:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 --2- 172.21.15.106:0/513408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f3a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 --2- 172.21.15.106:0/513408968 >> v2:172.21.15.204:3300/0 conn(0x7ff60c15cb70 0x7ff60c06ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 -- 172.21.15.106:0/513408968 >> 172.21.15.106:0/513408968 conn(0x7ff60c0b2d50 msgr2=0x7ff60c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 -- 172.21.15.106:0/513408968 shutdown_connections 2024-04-26T21:48:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.613+0000 7ff614600640 1 -- 172.21.15.106:0/513408968 wait complete. 2024-04-26T21:48:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.614+0000 7ff614600640 1 Processor -- start 2024-04-26T21:48:15.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.615+0000 7ff614600640 1 -- start start 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff60c06c770 0x7ff60c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff60c15cb70 0x7ff60c072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c15c780 con 0x7ff60c15cfd0 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c15c600 con 0x7ff60c15cb70 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff614600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c0af8f0 con 0x7ff60c06c770 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57420/0 (socket says 172.21.15.106:57420) 2024-04-26T21:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 -- 172.21.15.106:0/3174965070 learned_addr learned my addr 172.21.15.106:0/3174965070 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 -- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff60c06c770 msgr2=0x7ff60c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff60c06c770 0x7ff60c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 -- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff60c15cb70 msgr2=0x7ff60c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff60c15cb70 0x7ff60c072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.616+0000 7ff612600640 1 -- 172.21.15.106:0/3174965070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c072a90 con 0x7ff60c15cfd0 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff612600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff604034040 tx=0x7ff604004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604041020 con 0x7ff60c15cfd0 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff604007780 con 0x7ff60c15cfd0 2024-04-26T21:48:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c072d70 con 0x7ff60c15cfd0 2024-04-26T21:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604036040 con 0x7ff60c15cfd0 2024-04-26T21:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.617+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c073330 con 0x7ff60c15cfd0 2024-04-26T21:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.618+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c069540 con 0x7ff60c15cfd0 2024-04-26T21:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.619+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff604044020 con 0x7ff60c15cfd0 2024-04-26T21:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.620+0000 7ff609600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 0x7ff5ec06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.620+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff6040acef0 con 0x7ff60c15cfd0 2024-04-26T21:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.620+0000 7ff611c00640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 0x7ff5ec06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.621+0000 7ff611c00640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 0x7ff5ec06ef60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:15.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.624+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff60404b030 con 0x7ff60c15cfd0 2024-04-26T21:48:15.646 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:15.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:15.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.817+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff60c0d0680 con 0x7ff60c15cfd0 2024-04-26T21:48:15.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.820+0000 7ff609600640 1 -- 172.21.15.106:0/3174965070 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16625 (secure 0 0 0) 0x7ff60407afb0 con 0x7ff60c15cfd0 2024-04-26T21:48:15.820 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:15.820 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":45,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:03.337384+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.822+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 msgr2=0x7ff5ec06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.822+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 0x7ff5ec06ef60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.822+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 msgr2=0x7ff60c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.822+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff604034040 tx=0x7ff604004a40 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 shutdown_connections 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c15cfd0 0x7ff60c06f250 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff60c15cb70 0x7ff60c072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff5ec06caa0 0x7ff5ec06ef60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 --2- 172.21.15.106:0/3174965070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff60c06c770 0x7ff60c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 >> 172.21.15.106:0/3174965070 conn(0x7ff60c0b2d50 msgr2=0x7ff60c0b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 shutdown_connections 2024-04-26T21:48:15.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:15.823+0000 7ff614600640 1 -- 172.21.15.106:0/3174965070 wait complete. 2024-04-26T21:48:15.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:59.554855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:48:00.766548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:15.842 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.036+0000 7f2d42c00640 1 Processor -- start 2024-04-26T21:48:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.037+0000 7f2d42c00640 1 -- start start 2024-04-26T21:48:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.038+0000 7f2d42c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c160d60 0x7f2d3c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.038+0000 7f2d42c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d3c161140 0x7f2d3c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.038+0000 7f2d42c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c0b4d20 con 0x7f2d3c146280 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.038+0000 7f2d42c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3c0b5000 con 0x7f2d3c161140 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.038+0000 7f2d42c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c0b52e0 con 0x7f2d3c160d60 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.039+0000 7f2d40e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d3c161140 0x7f2d3c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.039+0000 7f2d40e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d3c161140 0x7f2d3c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57434/0 (socket says 172.21.15.106:57434) 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.039+0000 7f2d40e00640 1 -- 172.21.15.106:0/2673979554 learned_addr learned my addr 172.21.15.106:0/2673979554 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.039+0000 7f2d41800640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c160d60 0x7f2d3c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.040+0000 7f2d40e00640 1 -- 172.21.15.106:0/2673979554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c160d60 msgr2=0x7f2d3c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.040+0000 7f2d40e00640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c160d60 0x7f2d3c0b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.040+0000 7f2d40e00640 1 -- 172.21.15.106:0/2673979554 >> v1:172.21.15.204:6789/0 conn(0x7f2d3c146280 legacy=0x7f2d3c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.040+0000 7f2d40e00640 1 -- 172.21.15.106:0/2673979554 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c0cf410 con 0x7f2d3c161140 2024-04-26T21:48:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.041+0000 7f2d40e00640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c161140 0x7f2d3c06dee0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009870 tx=0x7f2d2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef1ddd176702ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.042+0000 7f2d3be00640 1 -- 172.21.15.106:0/2673979554 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28037070 con 0x7f2d3c161140 2024-04-26T21:48:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.042+0000 7f2d3be00640 1 -- 172.21.15.106:0/2673979554 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d28032440 con 0x7f2d3c161140 2024-04-26T21:48:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.042+0000 7f2d3be00640 1 -- 172.21.15.106:0/2673979554 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28032810 con 0x7f2d3c161140 2024-04-26T21:48:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.042+0000 7f2d42c00640 1 -- 172.21.15.106:0/2673979554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c161140 msgr2=0x7f2d3c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.042+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c161140 0x7f2d3c06dee0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009870 tx=0x7f2d2802a270 comp rx=0 tx=0).stop 2024-04-26T21:48:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 -- 172.21.15.106:0/2673979554 shutdown_connections 2024-04-26T21:48:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c161140 0x7f2d3c06dee0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2673979554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c160d60 0x7f2d3c0b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 -- 172.21.15.106:0/2673979554 >> 172.21.15.106:0/2673979554 conn(0x7f2d3c0b2900 msgr2=0x7f2d3c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 -- 172.21.15.106:0/2673979554 shutdown_connections 2024-04-26T21:48:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.043+0000 7f2d42c00640 1 -- 172.21.15.106:0/2673979554 wait complete. 2024-04-26T21:48:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.044+0000 7f2d42c00640 1 Processor -- start 2024-04-26T21:48:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 -- start start 2024-04-26T21:48:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c160d60 0x7f2d3c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c161140 0x7f2d3c1f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c06e420 con 0x7f2d3c146280 2024-04-26T21:48:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0b6070 con 0x7f2d3c160d60 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.045+0000 7f2d42c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c06e5a0 con 0x7f2d3c161140 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57440/0 (socket says 172.21.15.106:57440) 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 -- 172.21.15.106:0/2477904074 learned_addr learned my addr 172.21.15.106:0/2477904074 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 -- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c161140 msgr2=0x7f2d3c1f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d42200640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c161140 0x7f2d3c1f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d40e00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c160d60 0x7f2d3c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c161140 0x7f2d3c1f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 -- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c160d60 msgr2=0x7f2d3c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c160d60 0x7f2d3c15dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.046+0000 7f2d41800640 1 -- 172.21.15.106:0/2477904074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c15b540 con 0x7f2d3c146280 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d41800640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000d800 tx=0x7f2d3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000b6c0 con 0x7f2d3c146280 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30019070 con 0x7f2d3c146280 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c15b820 con 0x7f2d3c146280 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000ba80 con 0x7f2d3c146280 2024-04-26T21:48:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.047+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c15bde0 con 0x7f2d3c146280 2024-04-26T21:48:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.048+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c0693c0 con 0x7f2d3c146280 2024-04-26T21:48:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.049+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2d3000bc20 con 0x7f2d3c146280 2024-04-26T21:48:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.051+0000 7f2d39600640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 0x7f2d1c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.051+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2d3008c330 con 0x7f2d3c146280 2024-04-26T21:48:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.051+0000 7f2d40e00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 0x7f2d1c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.052+0000 7f2d40e00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 0x7f2d1c06ef60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2d28034000 tx=0x7f2d2802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.056+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d3005a4e0 con 0x7f2d3c146280 2024-04-26T21:48:16.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.256+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d3c0d1820 con 0x7f2d3c146280 2024-04-26T21:48:16.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.256+0000 7f2d39600640 1 -- 172.21.15.106:0/2477904074 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f2d30059de0 con 0x7f2d3c146280 2024-04-26T21:48:16.256 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:48:16.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.259+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 msgr2=0x7f2d1c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.259+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 0x7f2d1c06ef60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2d28034000 tx=0x7f2d2802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:16.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.259+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 msgr2=0x7f2d3c15c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.259+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000d800 tx=0x7f2d3000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 shutdown_connections 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2d1c06caa0 0x7f2d1c06ef60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d3c161140 0x7f2d3c1f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c160d60 0x7f2d3c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 --2- 172.21.15.106:0/2477904074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3c146280 0x7f2d3c15c5e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 >> 172.21.15.106:0/2477904074 conn(0x7f2d3c0b2900 msgr2=0x7f2d3c145590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 shutdown_connections 2024-04-26T21:48:16.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.260+0000 7f2d42c00640 1 -- 172.21.15.106:0/2477904074 wait complete. 2024-04-26T21:48:16.279 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:16.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.483+0000 7f82fa000640 1 Processor -- start 2024-04-26T21:48:16.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.483+0000 7f82fa000640 1 -- start start 2024-04-26T21:48:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82fa000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82f4161140 0x7f82f406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82fa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f41466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82fa000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f40b4d20 con 0x7f82f4160d60 2024-04-26T21:48:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82fa000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f82f4146c30 con 0x7f82f4161140 2024-04-26T21:48:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82fa000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82f40b5000 con 0x7f82f4146280 2024-04-26T21:48:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.485+0000 7f82f3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82f4161140 0x7f82f406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.486+0000 7f82f3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82f4161140 0x7f82f406dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:59036/0 (socket says 172.21.15.106:59036) 2024-04-26T21:48:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.486+0000 7f82f3e00640 1 -- 172.21.15.106:0/2354421810 learned_addr learned my addr 172.21.15.106:0/2354421810 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.486+0000 7f82f9600640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f41466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.487+0000 7f82f3e00640 1 -- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 msgr2=0x7f82f41466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:16.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.487+0000 7f82f3e00640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f41466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.487+0000 7f82f3e00640 1 -- 172.21.15.106:0/2354421810 >> v1:172.21.15.204:6789/0 conn(0x7f82f4160d60 legacy=0x7f82f40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.487+0000 7f82f3e00640 1 -- 172.21.15.106:0/2354421810 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f40b52e0 con 0x7f82f4161140 2024-04-26T21:48:16.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.488+0000 7f82f3e00640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4161140 0x7f82f406dee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098a0 tx=0x7f82e002a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432700d2deb33fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.488+0000 7f82f3400640 1 -- 172.21.15.106:0/2354421810 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0037070 con 0x7f82f4161140 2024-04-26T21:48:16.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.488+0000 7f82f3400640 1 -- 172.21.15.106:0/2354421810 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0032440 con 0x7f82f4161140 2024-04-26T21:48:16.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.488+0000 7f82f3400640 1 -- 172.21.15.106:0/2354421810 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0032810 con 0x7f82f4161140 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 -- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4161140 msgr2=0x7f82f406dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4161140 0x7f82f406dee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098a0 tx=0x7f82e002a2a0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 -- 172.21.15.106:0/2354421810 shutdown_connections 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f41466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 --2- 172.21.15.106:0/2354421810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4161140 0x7f82f406dee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 -- 172.21.15.106:0/2354421810 >> 172.21.15.106:0/2354421810 conn(0x7f82f40b2900 msgr2=0x7f82f40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 -- 172.21.15.106:0/2354421810 shutdown_connections 2024-04-26T21:48:16.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.489+0000 7f82fa000640 1 -- 172.21.15.106:0/2354421810 wait complete. 2024-04-26T21:48:16.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.491+0000 7f82fa000640 1 Processor -- start 2024-04-26T21:48:16.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.491+0000 7f82fa000640 1 -- start start 2024-04-26T21:48:16.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f40cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4160d60 0x7f82f4143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82f406e420 con 0x7f82f4161140 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40cf6f0 con 0x7f82f4160d60 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82fa000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82f406e5a0 con 0x7f82f4146280 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82f9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82f9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57450/0 (socket says 172.21.15.106:57450) 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.492+0000 7f82f9600640 1 -- 172.21.15.106:0/2901518207 learned_addr learned my addr 172.21.15.106:0/2901518207 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f3e00640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4160d60 0x7f82f4143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 -- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 msgr2=0x7f82f40cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f40cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 -- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4160d60 msgr2=0x7f82f4143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4160d60 0x7f82f4143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 -- 172.21.15.106:0/2901518207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f4143ca0 con 0x7f82f4161140 2024-04-26T21:48:16.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f9600640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00efc0 tx=0x7f82ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018020 con 0x7f82f4161140 2024-04-26T21:48:16.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec009b10 con 0x7f82f4161140 2024-04-26T21:48:16.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0146c0 con 0x7f82f4161140 2024-04-26T21:48:16.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.493+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41f4d60 con 0x7f82f4161140 2024-04-26T21:48:16.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.494+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41f5320 con 0x7f82f4161140 2024-04-26T21:48:16.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.495+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f40693c0 con 0x7f82f4161140 2024-04-26T21:48:16.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.496+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f82ec005d10 con 0x7f82f4161140 2024-04-26T21:48:16.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.497+0000 7f82f0c00640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 0x7f82d406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.497+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f82ec08c990 con 0x7f82f4161140 2024-04-26T21:48:16.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.497+0000 7f82f8c00640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 0x7f82d406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.498+0000 7f82f8c00640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 0x7f82d406ef60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f60 tx=0x7f82e80079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.501+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82ec022070 con 0x7f82f4161140 2024-04-26T21:48:16.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.699+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82f406f180 con 0x7f82f4161140 2024-04-26T21:48:16.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.700+0000 7f82f0c00640 1 -- 172.21.15.106:0/2901518207 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f82ec010040 con 0x7f82f4161140 2024-04-26T21:48:16.700 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.702+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 msgr2=0x7f82d406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.702+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 0x7f82d406ef60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f60 tx=0x7f82e80079d0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.702+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 msgr2=0x7f82f4141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.702+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00efc0 tx=0x7f82ec00c490 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 shutdown_connections 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82f4161140 0x7f82f4141640 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4160d60 0x7f82f4143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f82d406caa0 0x7f82d406ef60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 --2- 172.21.15.106:0/2901518207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82f4146280 0x7f82f40cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 >> 172.21.15.106:0/2901518207 conn(0x7f82f40b2900 msgr2=0x7f82f40730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 shutdown_connections 2024-04-26T21:48:16.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.703+0000 7f82fa000640 1 -- 172.21.15.106:0/2901518207 wait complete. 2024-04-26T21:48:16.722 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:16.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.916+0000 7f6b33200640 1 Processor -- start 2024-04-26T21:48:16.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.916+0000 7f6b33200640 1 -- start start 2024-04-26T21:48:16.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.917+0000 7f6b33200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6b341467a0 0x7f6b3415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.917+0000 7f6b33200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6b340b5ca0 0x7f6b340d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.917+0000 7f6b33200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3415e3e0 con 0x7f6b340d12a0 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.917+0000 7f6b33200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6b34146b80 con 0x7f6b340b5ca0 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.917+0000 7f6b33200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6b34161730 con 0x7f6b341467a0 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b2be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6b340b5ca0 0x7f6b340d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b32800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6b340d12a0 0x7f6b3406de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39932/0 (socket says 172.21.15.106:39932) 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b32800640 1 -- 172.21.15.106:0/3579939539 learned_addr learned my addr 172.21.15.106:0/3579939539 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:16.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b31e00640 1 --2- 172.21.15.106:0/3579939539 >> v2:172.21.15.204:3300/0 conn(0x7f6b341467a0 0x7f6b3415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b2be00640 1 -- 172.21.15.106:0/3579939539 >> v2:172.21.15.204:3300/0 conn(0x7f6b341467a0 msgr2=0x7f6b3415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b2be00640 1 --2- 172.21.15.106:0/3579939539 >> v2:172.21.15.204:3300/0 conn(0x7f6b341467a0 0x7f6b3415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b2be00640 1 -- 172.21.15.106:0/3579939539 >> v1:172.21.15.106:6789/0 conn(0x7f6b340d12a0 legacy=0x7f6b3406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b2be00640 1 -- 172.21.15.106:0/3579939539 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b34160b20 con 0x7f6b340b5ca0 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.918+0000 7f6b31400640 1 -- 172.21.15.106:0/3579939539 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557220806 0 0) 0x7f6b2c001490 con 0x7f6b340d12a0 2024-04-26T21:48:16.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.919+0000 7f6b2be00640 1 --2- 172.21.15.106:0/3579939539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b340b5ca0 0x7f6b340d0d60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b34074270 tx=0x7f6b1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4db2100e95b519 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b31400640 1 -- 172.21.15.106:0/3579939539 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c037070 con 0x7f6b340b5ca0 2024-04-26T21:48:16.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b31400640 1 -- 172.21.15.106:0/3579939539 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c032440 con 0x7f6b340b5ca0 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b31400640 1 -- 172.21.15.106:0/3579939539 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c032810 con 0x7f6b340b5ca0 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 -- 172.21.15.106:0/3579939539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b340b5ca0 msgr2=0x7f6b340d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 --2- 172.21.15.106:0/3579939539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b340b5ca0 0x7f6b340d0d60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b34074270 tx=0x7f6b1c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 -- 172.21.15.106:0/3579939539 shutdown_connections 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 --2- 172.21.15.106:0/3579939539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b340b5ca0 0x7f6b340d0d60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 --2- 172.21.15.106:0/3579939539 >> v2:172.21.15.204:3300/0 conn(0x7f6b341467a0 0x7f6b3415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.920+0000 7f6b33200640 1 -- 172.21.15.106:0/3579939539 >> 172.21.15.106:0/3579939539 conn(0x7f6b3406c530 msgr2=0x7f6b3406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.921+0000 7f6b33200640 1 -- 172.21.15.106:0/3579939539 shutdown_connections 2024-04-26T21:48:16.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.921+0000 7f6b33200640 1 -- 172.21.15.106:0/3579939539 wait complete. 2024-04-26T21:48:16.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.922+0000 7f6b33200640 1 Processor -- start 2024-04-26T21:48:16.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.923+0000 7f6b33200640 1 -- start start 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.923+0000 7f6b33200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b340b5ca0 0x7f6b341f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.923+0000 7f6b33200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b340d12a0 0x7f6b3415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.923+0000 7f6b33200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b33200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341610e0 con 0x7f6b341467a0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b33200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340d16d0 con 0x7f6b340b5ca0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b33200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6b341613c0 con 0x7f6b340d12a0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57466/0 (socket says 172.21.15.106:57466) 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 -- 172.21.15.106:0/3302867988 learned_addr learned my addr 172.21.15.106:0/3302867988 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b31e00640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b340b5ca0 0x7f6b341f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b2be00640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b340d12a0 0x7f6b3415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 -- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b340d12a0 msgr2=0x7f6b3415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b340d12a0 0x7f6b3415e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 -- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b340b5ca0 msgr2=0x7f6b341f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b340b5ca0 0x7f6b341f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.924+0000 7f6b32800640 1 -- 172.21.15.106:0/3302867988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3413da40 con 0x7f6b341467a0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b32800640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c7f0 tx=0x7f6b2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c018070 con 0x7f6b341467a0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c00a5e0 con 0x7f6b341467a0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00aa10 con 0x7f6b341467a0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3413dd20 con 0x7f6b341467a0 2024-04-26T21:48:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.925+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b340721b0 con 0x7f6b341467a0 2024-04-26T21:48:16.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.926+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b340693c0 con 0x7f6b341467a0 2024-04-26T21:48:16.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.927+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6b2c00abb0 con 0x7f6b341467a0 2024-04-26T21:48:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.928+0000 7f6b2a000640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 0x7f6b1006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.928+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2c08b300 con 0x7f6b341467a0 2024-04-26T21:48:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.928+0000 7f6b31e00640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 0x7f6b1006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.929+0000 7f6b31e00640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 0x7f6b1006ef90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b24006fd0 tx=0x7f6b24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:16.933+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b3415e3e0 con 0x7f6b341467a0 2024-04-26T21:48:17.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.136+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b3406f7a0 con 0x7f6b341467a0 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.137+0000 7f6b2a000640 1 -- 172.21.15.106:0/3302867988 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f6b2c059540 con 0x7f6b341467a0 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 msgr2=0x7f6b1006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 0x7f6b1006ef90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b24006fd0 tx=0x7f6b24008040 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 msgr2=0x7f6b3415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c7f0 tx=0x7f6b2c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 shutdown_connections 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b341467a0 0x7f6b3415eb00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6b1006cad0 0x7f6b1006ef90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b340d12a0 0x7f6b3415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 --2- 172.21.15.106:0/3302867988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b340b5ca0 0x7f6b341f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 >> 172.21.15.106:0/3302867988 conn(0x7f6b3406c530 msgr2=0x7f6b340b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 shutdown_connections 2024-04-26T21:48:17.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.140+0000 7f6b33200640 1 -- 172.21.15.106:0/3302867988 wait complete. 2024-04-26T21:48:17.160 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:17.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.360+0000 7f3027200640 1 Processor -- start 2024-04-26T21:48:17.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.360+0000 7f3027200640 1 -- start start 2024-04-26T21:48:17.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.361+0000 7f3027200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3028146680 0x7f30280b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.361+0000 7f3027200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f30280d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3027200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3028146a60 con 0x7f3028160930 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3027200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f302806e970 con 0x7f3028146680 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3027200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f30280d17f0 con 0x7f30280d0e70 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3026800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3028160930 0x7f3028160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39934/0 (socket says 172.21.15.106:39934) 2024-04-26T21:48:17.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3026800640 1 -- 172.21.15.106:0/1135209784 learned_addr learned my addr 172.21.15.106:0/1135209784 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3025e00640 1 --2- 172.21.15.106:0/1135209784 >> v2:172.21.15.204:3300/0 conn(0x7f3028146680 0x7f30280b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.362+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473125176 0 0) 0x7f3028146a60 con 0x7f3028160930 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f300c003520 con 0x7f3028160930 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3025400640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f30280d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3810601334 0 0) 0x7f300c003520 con 0x7f3028160930 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 msgr2=0x7f30280d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f30280d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 >> v2:172.21.15.204:3300/0 conn(0x7f3028146680 msgr2=0x7f30280b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 --2- 172.21.15.106:0/1135209784 >> v2:172.21.15.204:3300/0 conn(0x7f3028146680 0x7f30280b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302806dd60 con 0x7f3028160930 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f30200019f0 con 0x7f3028160930 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 >> v1:172.21.15.106:6789/0 conn(0x7f3028160930 legacy=0x7f3028160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f300c0039b0 0x7f300c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f300c004800 0x7f300c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f300c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30280d17f0 con 0x7f300c004800 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f300c00b9f0 con 0x7f300c0039b0 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f300c00bbf0 con 0x7f30280d0e70 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.363+0000 7f3025400640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f300c004800 0x7f300c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3026800640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f300c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025400640 1 -- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 msgr2=0x7f300c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025400640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30280d0e70 0x7f300c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025400640 1 -- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f300c0039b0 msgr2=0x7f300c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025e00640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f300c0039b0 0x7f300c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025400640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f300c0039b0 0x7f300c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.364+0000 7f3025400640 1 -- 172.21.15.106:0/1135209784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c003520 con 0x7f300c004800 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.365+0000 7f3025400640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f300c004800 0x7f300c004c80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f301400ba00 tx=0x7f300c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5577aa6af397d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.365+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014030070 con 0x7f300c004800 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.365+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301402ad80 con 0x7f300c004800 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.365+0000 7f3024a00640 1 -- 172.21.15.106:0/1135209784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301400b600 con 0x7f300c004800 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 -- 172.21.15.106:0/1135209784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f300c004800 msgr2=0x7f300c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 --2- 172.21.15.106:0/1135209784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f300c004800 0x7f300c004c80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f301400ba00 tx=0x7f300c005830 comp rx=0 tx=0).stop 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3026800640 1 -- 172.21.15.106:0/1135209784 reap_dead start 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 -- 172.21.15.106:0/1135209784 shutdown_connections 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 -- 172.21.15.106:0/1135209784 >> 172.21.15.106:0/1135209784 conn(0x7f302806c740 msgr2=0x7f302806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 -- 172.21.15.106:0/1135209784 shutdown_connections 2024-04-26T21:48:17.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.366+0000 7f3027200640 1 -- 172.21.15.106:0/1135209784 wait complete. 2024-04-26T21:48:17.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.368+0000 7f3027200640 1 Processor -- start 2024-04-26T21:48:17.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.368+0000 7f3027200640 1 -- start start 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f300c004800 0x7f302815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f302813b8e0 0x7f3028140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f302806e600 con 0x7f3028141280 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f302806e320 con 0x7f302813b8e0 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3027200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f30280cd6a0 con 0x7f300c004800 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3026800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3026800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57482/0 (socket says 172.21.15.106:57482) 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.369+0000 7f3026800640 1 -- 172.21.15.106:0/2811135180 learned_addr learned my addr 172.21.15.106:0/2811135180 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3025e00640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f300c004800 0x7f302815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3025400640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f302813b8e0 0x7f3028140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 -- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f300c004800 msgr2=0x7f302815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f300c004800 0x7f302815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 -- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f302813b8e0 msgr2=0x7f3028140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f302813b8e0 0x7f3028140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 -- 172.21.15.106:0/2811135180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302813d4b0 con 0x7f3028141280 2024-04-26T21:48:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3026800640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3020000f80 tx=0x7f302000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302000de50 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302000bb20 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020007110 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.370+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302813d730 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.371+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302813dcf0 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.372+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30280693c0 con 0x7f3028141280 2024-04-26T21:48:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.373+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f302001b060 con 0x7f3028141280 2024-04-26T21:48:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.373+0000 7f301e000640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 0x7f300806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.374+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f302008bf60 con 0x7f3028141280 2024-04-26T21:48:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.374+0000 7f3025e00640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 0x7f300806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.374+0000 7f3025e00640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 0x7f300806ef90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.378+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f302813d4b0 con 0x7f3028141280 2024-04-26T21:48:17.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.595+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f302806cf40 con 0x7f3028141280 2024-04-26T21:48:17.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.596+0000 7f301e000640 1 -- 172.21.15.106:0/2811135180 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f3020059ff0 con 0x7f3028141280 2024-04-26T21:48:17.597 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:48:17.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.598+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 msgr2=0x7f300806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 0x7f300806ef90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).stop 2024-04-26T21:48:17.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 msgr2=0x7f3028072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3020000f80 tx=0x7f302000b4a0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 shutdown_connections 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3028141280 0x7f3028072e90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f302813b8e0 0x7f3028140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f300806cad0 0x7f300806ef90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 --2- 172.21.15.106:0/2811135180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f300c004800 0x7f302815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 >> 172.21.15.106:0/2811135180 conn(0x7f302806c740 msgr2=0x7f302815d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 shutdown_connections 2024-04-26T21:48:17.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.599+0000 7f3027200640 1 -- 172.21.15.106:0/2811135180 wait complete. 2024-04-26T21:48:17.619 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:48:17.620 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:17.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:17.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.917+0000 7fcfe7200640 1 Processor -- start 2024-04-26T21:48:17.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.917+0000 7fcfe7200640 1 -- start start 2024-04-26T21:48:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.918+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 0x7fcfe80d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.918+0000 7fcfe7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe7200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe8146a60 con 0x7fcfe8146680 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe806e970 con 0x7fcfe80b3b20 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe80b60a0 con 0x7fcfe80d0e70 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 0x7fcfe80d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 0x7fcfe80d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.106:45730/0 (socket says 172.21.15.106:45730) 2024-04-26T21:48:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe5e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fcfe8146680 0x7fcfe80b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:41876/0 (socket says 172.21.15.106:41876) 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfe6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:44854/0 (socket says 172.21.15.106:44854) 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.919+0000 7fcfdfe00640 1 -- 172.21.15.106:0/1790586533 learned_addr learned my addr 172.21.15.106:0/1790586533 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.920+0000 7fcfe6800640 1 -- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 msgr2=0x7fcfe80d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.920+0000 7fcfe6800640 1 --2- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 0x7fcfe80d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.920+0000 7fcfe6800640 1 -- 172.21.15.106:0/1790586533 >> v1:172.21.15.204:6789/0 conn(0x7fcfe8146680 legacy=0x7fcfe80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.920+0000 7fcfe6800640 1 -- 172.21.15.106:0/1790586533 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe806dd60 con 0x7fcfe80b3b20 2024-04-26T21:48:17.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.921+0000 7fcfe6800640 1 --2- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009d90 tx=0x7fcfe002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddea247af65cf0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.922+0000 7fcfe5400640 1 -- 172.21.15.106:0/1790586533 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007740 con 0x7fcfe80b3b20 2024-04-26T21:48:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.922+0000 7fcfe5400640 1 -- 172.21.15.106:0/1790586533 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe00078e0 con 0x7fcfe80b3b20 2024-04-26T21:48:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.922+0000 7fcfe5400640 1 -- 172.21.15.106:0/1790586533 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007cb0 con 0x7fcfe80b3b20 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 -- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 msgr2=0x7fcfe80b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009d90 tx=0x7fcfe002a660 comp rx=0 tx=0).stop 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 -- 172.21.15.106:0/1790586533 shutdown_connections 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe80b3fb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1790586533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe80d0e70 0x7fcfe80d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 -- 172.21.15.106:0/1790586533 >> 172.21.15.106:0/1790586533 conn(0x7fcfe806c740 msgr2=0x7fcfe806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 -- 172.21.15.106:0/1790586533 shutdown_connections 2024-04-26T21:48:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.923+0000 7fcfe7200640 1 -- 172.21.15.106:0/1790586533 wait complete. 2024-04-26T21:48:17.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.925+0000 7fcfe7200640 1 Processor -- start 2024-04-26T21:48:17.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.925+0000 7fcfe7200640 1 -- start start 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe81f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe8146680 0x7fcfe813b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe806e320 con 0x7fcfe80d0e70 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d17f0 con 0x7fcfe80b3b20 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.926+0000 7fcfe7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe806e600 con 0x7fcfe8146680 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48500/0 (socket says 172.21.15.106:48500) 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 -- 172.21.15.106:0/1351839400 learned_addr learned my addr 172.21.15.106:0/1351839400 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfe5e00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe81f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfe6800640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe8146680 0x7fcfe813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 -- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe8146680 msgr2=0x7fcfe813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe8146680 0x7fcfe813b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 -- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 msgr2=0x7fcfe81f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe81f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.927+0000 7fcfdfe00640 1 -- 172.21.15.106:0/1351839400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe80734a0 con 0x7fcfe80d0e70 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfdfe00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fd0 tx=0x7fcfd000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0017070 con 0x7fcfe80d0e70 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd000c940 con 0x7fcfe80d0e70 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000cd10 con 0x7fcfe80d0e70 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe80ca8f0 con 0x7fcfe80d0e70 2024-04-26T21:48:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.928+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe80caeb0 con 0x7fcfe80d0e70 2024-04-26T21:48:17.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.929+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe80693c0 con 0x7fcfe80d0e70 2024-04-26T21:48:17.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.930+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fcfd001a070 con 0x7fcfe80d0e70 2024-04-26T21:48:17.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.931+0000 7fcfde000640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.931+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcfd008c170 con 0x7fcfe80d0e70 2024-04-26T21:48:17.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.931+0000 7fcfe5e00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.932+0000 7fcfe5e00640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 0x7fcfc406ef60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8000f60 tx=0x7fcfd8007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:17.936+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcfd005a310 con 0x7fcfe80d0e70 2024-04-26T21:48:18.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.136+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfe80d15d0 con 0x7fcfe80d0e70 2024-04-26T21:48:18.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.138+0000 7fcfde000640 1 -- 172.21.15.106:0/1351839400 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16625 (secure 0 0 0) 0x7fcfd0059c10 con 0x7fcfe80d0e70 2024-04-26T21:48:18.140 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:18.140 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":45,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:03.337384+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:18.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 msgr2=0x7fcfc406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 0x7fcfc406ef60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8000f60 tx=0x7fcfd8007ce0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 msgr2=0x7fcfe813af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fd0 tx=0x7fcfd000ed40 comp rx=0 tx=0).stop 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 shutdown_connections 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fcfc406caa0 0x7fcfc406ef60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfe8146680 0x7fcfe813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe80d0e70 0x7fcfe813af40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 --2- 172.21.15.106:0/1351839400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfe80b3b20 0x7fcfe81f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 >> 172.21.15.106:0/1351839400 conn(0x7fcfe806c740 msgr2=0x7fcfe8160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 shutdown_connections 2024-04-26T21:48:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.141+0000 7fcfe7200640 1 -- 172.21.15.106:0/1351839400 wait complete. 2024-04-26T21:48:18.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:59.554855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:48:00.766548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:18.160 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:18.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.361+0000 7f2362800640 1 Processor -- start 2024-04-26T21:48:18.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.361+0000 7f2362800640 1 -- start start 2024-04-26T21:48:18.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2362800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235c1603e0 0x7f235c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2362800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c0b6810 con 0x7f235c15ff30 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2362800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c0d1640 con 0x7f235c159770 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2362800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f235c15faa0 con 0x7f235c1603e0 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2360a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f235c15ff30 0x7f235c06e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56690/0 (socket says 172.21.15.106:56690) 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2360a00640 1 -- 172.21.15.106:0/2032877366 learned_addr learned my addr 172.21.15.106:0/2032877366 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593941835 0 0) 0x7f235c0b6810 con 0x7f235c15ff30 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2344003520 con 0x7f235c15ff30 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.364+0000 7f2361e00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235c1603e0 0x7f235c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2857297097 0 0) 0x7f2344003520 con 0x7f235c15ff30 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235c1603e0 msgr2=0x7f235c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235c1603e0 0x7f235c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 >> v1:172.21.15.204:6789/0 conn(0x7f235c159770 legacy=0x7f235c163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c0d1920 con 0x7f235c15ff30 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2348001690 con 0x7f235c15ff30 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 >> v1:172.21.15.106:6789/0 conn(0x7f235c15ff30 legacy=0x7f235c06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344003a50 0x7f2344003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344004800 0x7f2344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2344005210 0x7f234400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f235c15faa0 con 0x7f2344005210 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f234400c140 con 0x7f2344004800 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f234400c340 con 0x7f2344003a50 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f2361e00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2344005210 0x7f234400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f2361400640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344003a50 0x7f2344003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f2360a00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344004800 0x7f2344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.365+0000 7f2361e00640 1 -- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344003a50 msgr2=0x7f2344003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.366+0000 7f2361e00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344003a50 0x7f2344003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.366+0000 7f2361e00640 1 -- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344004800 msgr2=0x7f2344004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:18.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.366+0000 7f2361e00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344004800 0x7f2344004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.366+0000 7f2361e00640 1 -- 172.21.15.106:0/2032877366 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2344003520 con 0x7f2344005210 2024-04-26T21:48:18.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.366+0000 7f2360a00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344004800 0x7f2344004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.367+0000 7f2361e00640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2344005210 0x7f234400b9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f235c1607c0 tx=0x7f2344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de7dc15aaa5f539 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.367+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354030070 con 0x7f2344005210 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.367+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235402acc0 con 0x7f2344005210 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.367+0000 7f235be00640 1 -- 172.21.15.106:0/2032877366 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235400b550 con 0x7f2344005210 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 -- 172.21.15.106:0/2032877366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2344005210 msgr2=0x7f234400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 --2- 172.21.15.106:0/2032877366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2344005210 0x7f234400b9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f235c1607c0 tx=0x7f2344005830 comp rx=0 tx=0).stop 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2361e00640 1 -- 172.21.15.106:0/2032877366 reap_dead start 2024-04-26T21:48:18.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 -- 172.21.15.106:0/2032877366 shutdown_connections 2024-04-26T21:48:18.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 -- 172.21.15.106:0/2032877366 >> 172.21.15.106:0/2032877366 conn(0x7f235c06c350 msgr2=0x7f235c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:18.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 -- 172.21.15.106:0/2032877366 shutdown_connections 2024-04-26T21:48:18.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.368+0000 7f2362800640 1 -- 172.21.15.106:0/2032877366 wait complete. 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.370+0000 7f2362800640 1 Processor -- start 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.370+0000 7f2362800640 1 -- start start 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344005210 0x7f235c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235c155390 0x7f235c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f235c159cd0 con 0x7f235c1505a0 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f235c159b50 con 0x7f235c155390 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2362800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f235c06eed0 con 0x7f2344005210 2024-04-26T21:48:18.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48516/0 (socket says 172.21.15.106:48516) 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 -- 172.21.15.106:0/1686506218 learned_addr learned my addr 172.21.15.106:0/1686506218 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2361400640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344005210 0x7f235c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 -- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344005210 msgr2=0x7f235c13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2361e00640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235c155390 0x7f235c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344005210 0x7f235c13d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 -- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235c155390 msgr2=0x7f235c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235c155390 0x7f235c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.371+0000 7f2360a00640 1 -- 172.21.15.106:0/1686506218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1510d0 con 0x7f235c1505a0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2360a00640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2348001010 tx=0x7f234800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348019070 con 0x7f235c1505a0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23480074e0 con 0x7f235c1505a0 2024-04-26T21:48:18.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348004a00 con 0x7f235c1505a0 2024-04-26T21:48:18.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1513b0 con 0x7f235c1505a0 2024-04-26T21:48:18.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.372+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c13d8b0 con 0x7f235c1505a0 2024-04-26T21:48:18.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.373+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c069ad0 con 0x7f235c1505a0 2024-04-26T21:48:18.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.374+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f234801c070 con 0x7f235c1505a0 2024-04-26T21:48:18.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.375+0000 7f2359600640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 0x7f233c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.376+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f234808b910 con 0x7f235c1505a0 2024-04-26T21:48:18.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.376+0000 7f2361400640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 0x7f233c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.376+0000 7f2361400640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 0x7f233c06ef90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2350000f80 tx=0x7f2350009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.380+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2348059410 con 0x7f235c1505a0 2024-04-26T21:48:18.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.566+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f235c0754b0 con 0x7f235c1505a0 2024-04-26T21:48:18.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.566+0000 7f2359600640 1 -- 172.21.15.106:0/1686506218 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f234805d710 con 0x7f235c1505a0 2024-04-26T21:48:18.567 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:48:18.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 msgr2=0x7f233c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 0x7f233c06ef90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2350000f80 tx=0x7f2350009040 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 msgr2=0x7f235c150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2348001010 tx=0x7f234800dd40 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 shutdown_connections 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235c155390 0x7f235c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f235c1505a0 0x7f235c150a20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f233c06cad0 0x7f233c06ef90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 --2- 172.21.15.106:0/1686506218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2344005210 0x7f235c13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 >> 172.21.15.106:0/1686506218 conn(0x7f235c06c350 msgr2=0x7f235c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 shutdown_connections 2024-04-26T21:48:18.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.569+0000 7f2362800640 1 -- 172.21.15.106:0/1686506218 wait complete. 2024-04-26T21:48:18.587 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:18.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.786+0000 7f67ed000640 1 Processor -- start 2024-04-26T21:48:18.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.787+0000 7f67ed000640 1 -- start start 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.788+0000 7f67ed000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67e815f2a0 0x7f67e8163890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.788+0000 7f67ed000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80d0070 0x7f67e8161e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.788+0000 7f67ed000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e8160af0 con 0x7f67e80cfc10 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.788+0000 7f67ed000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f67e80d3fe0 con 0x7f67e815f2a0 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.788+0000 7f67ed000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67e80d0520 con 0x7f67e80d0070 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67e815f2a0 0x7f67e8163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67e815f2a0 0x7f67e8163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48524/0 (socket says 172.21.15.106:48524) 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 -- 172.21.15.106:0/828716785 learned_addr learned my addr 172.21.15.106:0/828716785 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7e00640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80d0070 0x7f67e8161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 -- 172.21.15.106:0/828716785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80d0070 msgr2=0x7f67e8161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80d0070 0x7f67e8161e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 -- 172.21.15.106:0/828716785 >> v1:172.21.15.204:6789/0 conn(0x7f67e80cfc10 legacy=0x7f67e80d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:18.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.789+0000 7f67e7400640 1 -- 172.21.15.106:0/828716785 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e80d0800 con 0x7f67e815f2a0 2024-04-26T21:48:18.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.790+0000 7f67e7400640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8163890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009870 tx=0x7f67d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98d9bf1b9f7309 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.790+0000 7f67e6000640 1 -- 172.21.15.106:0/828716785 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8037070 con 0x7f67e815f2a0 2024-04-26T21:48:18.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67e6000640 1 -- 172.21.15.106:0/828716785 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d8032440 con 0x7f67e815f2a0 2024-04-26T21:48:18.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67e6000640 1 -- 172.21.15.106:0/828716785 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8032810 con 0x7f67e815f2a0 2024-04-26T21:48:18.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 -- 172.21.15.106:0/828716785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 msgr2=0x7f67e8163890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8163890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009870 tx=0x7f67d802a210 comp rx=0 tx=0).stop 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 -- 172.21.15.106:0/828716785 shutdown_connections 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80d0070 0x7f67e8161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 --2- 172.21.15.106:0/828716785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8163890 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 -- 172.21.15.106:0/828716785 >> 172.21.15.106:0/828716785 conn(0x7f67e809e290 msgr2=0x7f67e80d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 -- 172.21.15.106:0/828716785 shutdown_connections 2024-04-26T21:48:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.791+0000 7f67ed000640 1 -- 172.21.15.106:0/828716785 wait complete. 2024-04-26T21:48:18.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.793+0000 7f67ed000640 1 Processor -- start 2024-04-26T21:48:18.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.793+0000 7f67ed000640 1 -- start start 2024-04-26T21:48:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80cfc10 0x7f67e80b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67e80d0070 0x7f67e815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80d2120 con 0x7f67e815f2a0 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80d0dc0 con 0x7f67e80d0070 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67ed000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67e80d22a0 con 0x7f67e80cfc10 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67e7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67e7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48536/0 (socket says 172.21.15.106:48536) 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.794+0000 7f67e7e00640 1 -- 172.21.15.106:0/153633374 learned_addr learned my addr 172.21.15.106:0/153633374 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7400640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80cfc10 0x7f67e80b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e6a00640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67e80d0070 0x7f67e815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 -- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80cfc10 msgr2=0x7f67e80b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80cfc10 0x7f67e80b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 -- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67e80d0070 msgr2=0x7f67e815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67e80d0070 0x7f67e815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 -- 172.21.15.106:0/153633374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e80ba7c0 con 0x7f67e815f2a0 2024-04-26T21:48:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67e7e00640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007e60 tx=0x7f67e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0014420 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e0019070 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0014820 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.795+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e80b8790 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.796+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e8159f90 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.797+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e80cf330 con 0x7f67e815f2a0 2024-04-26T21:48:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.798+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f67e00149c0 con 0x7f67e815f2a0 2024-04-26T21:48:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.799+0000 7f67df400640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 0x7f67c806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.799+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f67e008c630 con 0x7f67e815f2a0 2024-04-26T21:48:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.799+0000 7f67e7400640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 0x7f67c806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.800+0000 7f67e7400640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 0x7f67c806ef90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f67d8034000 tx=0x7f67d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:18.803+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67e80cf330 con 0x7f67e815f2a0 2024-04-26T21:48:19.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.003+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67e8069ad0 con 0x7f67e815f2a0 2024-04-26T21:48:19.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.004+0000 7f67df400640 1 -- 172.21.15.106:0/153633374 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f67e005a870 con 0x7f67e815f2a0 2024-04-26T21:48:19.005 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:48:19.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.006+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 msgr2=0x7f67c806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.006+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 0x7f67c806ef90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f67d8034000 tx=0x7f67d802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.006+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 msgr2=0x7f67e8159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.006+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007e60 tx=0x7f67e000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 shutdown_connections 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e815f2a0 0x7f67e8159a50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67e80d0070 0x7f67e815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f67c806cad0 0x7f67c806ef90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 --2- 172.21.15.106:0/153633374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67e80cfc10 0x7f67e80b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 >> 172.21.15.106:0/153633374 conn(0x7f67e809e290 msgr2=0x7f67e80d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 shutdown_connections 2024-04-26T21:48:19.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.007+0000 7f67ed000640 1 -- 172.21.15.106:0/153633374 wait complete. 2024-04-26T21:48:19.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:19.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.230+0000 7f90edc00640 1 Processor -- start 2024-04-26T21:48:19.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.230+0000 7f90edc00640 1 -- start start 2024-04-26T21:48:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.231+0000 7f90edc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 0x7f90e815ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.231+0000 7f90edc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f90e80d0d60 0x7f90e80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90edc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e8072170 con 0x7f90e80ae8c0 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90edc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f90e8072d80 con 0x7f90e80d0d60 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90edc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f90e8072450 con 0x7f90e8159530 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90e7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 0x7f90e815ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90ed200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f90e80ae8c0 0x7f90e80aed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:41910/0 (socket says 172.21.15.106:41910) 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.232+0000 7f90ed200640 1 -- 172.21.15.106:0/2346069157 learned_addr learned my addr 172.21.15.106:0/2346069157 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:19.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7400640 1 --2- 172.21.15.106:0/2346069157 >> v2:172.21.15.204:3300/0 conn(0x7f90e80d0d60 0x7f90e80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7400640 1 -- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 msgr2=0x7f90e815ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7400640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 0x7f90e815ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7400640 1 -- 172.21.15.106:0/2346069157 >> v1:172.21.15.204:6789/0 conn(0x7f90e80ae8c0 legacy=0x7f90e80aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7400640 1 -- 172.21.15.106:0/2346069157 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8072730 con 0x7f90e80d0d60 2024-04-26T21:48:19.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.233+0000 7f90e7e00640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 0x7f90e815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:48:19.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.234+0000 7f90e7400640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80d0d60 0x7f90e80d11e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97834593f8c0bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.235+0000 7f90e6a00640 1 -- 172.21.15.106:0/2346069157 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4037070 con 0x7f90e80d0d60 2024-04-26T21:48:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.235+0000 7f90e6a00640 1 -- 172.21.15.106:0/2346069157 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d4032440 con 0x7f90e80d0d60 2024-04-26T21:48:19.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.235+0000 7f90e6a00640 1 -- 172.21.15.106:0/2346069157 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4032810 con 0x7f90e80d0d60 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 -- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80d0d60 msgr2=0x7f90e80d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80d0d60 0x7f90e80d11e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402a180 comp rx=0 tx=0).stop 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 -- 172.21.15.106:0/2346069157 shutdown_connections 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80d0d60 0x7f90e80d11e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 --2- 172.21.15.106:0/2346069157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e8159530 0x7f90e815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 -- 172.21.15.106:0/2346069157 >> 172.21.15.106:0/2346069157 conn(0x7f90e806c530 msgr2=0x7f90e806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 -- 172.21.15.106:0/2346069157 shutdown_connections 2024-04-26T21:48:19.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.236+0000 7f90edc00640 1 -- 172.21.15.106:0/2346069157 wait complete. 2024-04-26T21:48:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.238+0000 7f90edc00640 1 Processor -- start 2024-04-26T21:48:19.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.238+0000 7f90edc00640 1 -- start start 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80ae8c0 0x7f90e81f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e80d0d60 0x7f90e814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8071e40 con 0x7f90e8159530 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81604c0 con 0x7f90e80ae8c0 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90edc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f90e8159910 con 0x7f90e80d0d60 2024-04-26T21:48:19.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90ed200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90ed200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48546/0 (socket says 172.21.15.106:48546) 2024-04-26T21:48:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90ed200640 1 -- 172.21.15.106:0/1526349283 learned_addr learned my addr 172.21.15.106:0/1526349283 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90e7e00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80ae8c0 0x7f90e81f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.239+0000 7f90e7400640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e80d0d60 0x7f90e814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 -- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e80d0d60 msgr2=0x7f90e814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e80d0d60 0x7f90e814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 -- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80ae8c0 msgr2=0x7f90e81f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80ae8c0 0x7f90e81f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 -- 172.21.15.106:0/1526349283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e81f5260 con 0x7f90e8159530 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90ed200640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f90e000cbc0 tx=0x7f90e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0017070 con 0x7f90e8159530 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e000ad40 con 0x7f90e8159530 2024-04-26T21:48:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0012600 con 0x7f90e8159530 2024-04-26T21:48:19.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.240+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e814ec90 con 0x7f90e8159530 2024-04-26T21:48:19.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.241+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e814f250 con 0x7f90e8159530 2024-04-26T21:48:19.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.242+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e80693c0 con 0x7f90e8159530 2024-04-26T21:48:19.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.243+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f90e001a070 con 0x7f90e8159530 2024-04-26T21:48:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.244+0000 7f90dfe00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 0x7f90c806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.244+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f90e008ba80 con 0x7f90e8159530 2024-04-26T21:48:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.244+0000 7f90e7e00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 0x7f90c806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.245+0000 7f90e7e00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 0x7f90c806ef60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009560 tx=0x7f90d8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.248+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90e008de20 con 0x7f90e8159530 2024-04-26T21:48:19.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.452+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90e80b2770 con 0x7f90e8159530 2024-04-26T21:48:19.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.455+0000 7f90dfe00640 1 -- 172.21.15.106:0/1526349283 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f90e81f5260 con 0x7f90e8159530 2024-04-26T21:48:19.456 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:48:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 msgr2=0x7f90c806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 0x7f90c806ef60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009560 tx=0x7f90d8009190 comp rx=0 tx=0).stop 2024-04-26T21:48:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 msgr2=0x7f90e8155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f90e000cbc0 tx=0x7f90e000a430 comp rx=0 tx=0).stop 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 shutdown_connections 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90e8159530 0x7f90e8155f90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f90c806caa0 0x7f90c806ef60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e80d0d60 0x7f90e814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 --2- 172.21.15.106:0/1526349283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e80ae8c0 0x7f90e81f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 >> 172.21.15.106:0/1526349283 conn(0x7f90e806c530 msgr2=0x7f90e80b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 shutdown_connections 2024-04-26T21:48:19.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.458+0000 7f90edc00640 1 -- 172.21.15.106:0/1526349283 wait complete. 2024-04-26T21:48:19.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:19.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.676+0000 7f3cdd000640 1 Processor -- start 2024-04-26T21:48:19.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.676+0000 7f3cdd000640 1 -- start start 2024-04-26T21:48:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.677+0000 7f3cdd000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cd815cdf0 0x7f3cd80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 0x7f3cd806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cdd000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3cd80b4f50 0x7f3cd806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cdd000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd80ae2c0 con 0x7f3cd815cdf0 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cdd000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd80b05b0 con 0x7f3cd80b4f50 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cdd000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd80af9a0 con 0x7f3cd80cd650 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cd815cdf0 0x7f3cd80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cd815cdf0 0x7f3cd80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48552/0 (socket says 172.21.15.106:48552) 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 -- 172.21.15.106:0/2230524616 learned_addr learned my addr 172.21.15.106:0/2230524616 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd6800640 1 --2- 172.21.15.106:0/2230524616 >> v2:172.21.15.204:3300/0 conn(0x7f3cd80b4f50 0x7f3cd806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5400640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 0x7f3cd806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 -- 172.21.15.106:0/2230524616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 msgr2=0x7f3cd806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 0x7f3cd806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 -- 172.21.15.106:0/2230524616 >> v2:172.21.15.204:3300/0 conn(0x7f3cd80b4f50 msgr2=0x7f3cd806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:19.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/2230524616 >> v2:172.21.15.204:3300/0 conn(0x7f3cd80b4f50 0x7f3cd806ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.678+0000 7f3cd5e00640 1 -- 172.21.15.106:0/2230524616 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd80afc80 con 0x7f3cd815cdf0 2024-04-26T21:48:19.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.679+0000 7f3cd5400640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 0x7f3cd806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.679+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd815cdf0 0x7f3cd80add80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b0520 tx=0x7f3cc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a90b4585a1e1a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.680+0000 7f3cd4a00640 1 -- 172.21.15.106:0/2230524616 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8037070 con 0x7f3cd815cdf0 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.680+0000 7f3cd4a00640 1 -- 172.21.15.106:0/2230524616 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8032440 con 0x7f3cd815cdf0 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.680+0000 7f3cd4a00640 1 -- 172.21.15.106:0/2230524616 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8032810 con 0x7f3cd815cdf0 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.680+0000 7f3cdd000640 1 -- 172.21.15.106:0/2230524616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd815cdf0 msgr2=0x7f3cd80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.680+0000 7f3cdd000640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd815cdf0 0x7f3cd80add80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b0520 tx=0x7f3cc802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 -- 172.21.15.106:0/2230524616 shutdown_connections 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 --2- 172.21.15.106:0/2230524616 >> v2:172.21.15.204:3300/0 conn(0x7f3cd80b4f50 0x7f3cd806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80cd650 0x7f3cd806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 --2- 172.21.15.106:0/2230524616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd815cdf0 0x7f3cd80add80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 -- 172.21.15.106:0/2230524616 >> 172.21.15.106:0/2230524616 conn(0x7f3cd80b2d50 msgr2=0x7f3cd80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 -- 172.21.15.106:0/2230524616 shutdown_connections 2024-04-26T21:48:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.681+0000 7f3cdd000640 1 -- 172.21.15.106:0/2230524616 wait complete. 2024-04-26T21:48:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.682+0000 7f3cdd000640 1 Processor -- start 2024-04-26T21:48:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.683+0000 7f3cdd000640 1 -- start start 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.683+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80b4f50 0x7f3cd814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.683+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.683+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 0x7f3cd809e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.683+0000 7f3cdd000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd815d1d0 con 0x7f3cd80cd650 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cdd000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b0240 con 0x7f3cd815cdf0 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cdd000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd815d350 con 0x7f3cd80b4f50 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48568/0 (socket says 172.21.15.106:48568) 2024-04-26T21:48:19.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 -- 172.21.15.106:0/3919102310 learned_addr learned my addr 172.21.15.106:0/3919102310 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd6800640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 0x7f3cd809e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 -- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80b4f50 msgr2=0x7f3cd814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80b4f50 0x7f3cd814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80b4f50 0x7f3cd814bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 -- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 msgr2=0x7f3cd809e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 0x7f3cd809e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd5400640 1 -- 172.21.15.106:0/3919102310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd814c460 con 0x7f3cd80cd650 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.684+0000 7f3cd6800640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 0x7f3cd809e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cd5400640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c7f0 tx=0x7f3cc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4015070 con 0x7f3cd80cd650 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4007820 con 0x7f3cd80cd650 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4007bd0 con 0x7f3cd80cd650 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd815e6d0 con 0x7f3cd80cd650 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.685+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd815ec90 con 0x7f3cd80cd650 2024-04-26T21:48:19.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.686+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8069540 con 0x7f3cd80cd650 2024-04-26T21:48:19.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.687+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3cc4018070 con 0x7f3cd80cd650 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.689+0000 7f3cce000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 0x7f3cb806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.689+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3cc408c120 con 0x7f3cd80cd650 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.689+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 0x7f3cb806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.690+0000 7f3cd5e00640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 0x7f3cb806ef60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8034000 tx=0x7f3cc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.693+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cc405a090 con 0x7f3cd80cd650 2024-04-26T21:48:19.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.905+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cd814bbb0 con 0x7f3cd80cd650 2024-04-26T21:48:19.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.906+0000 7f3cce000640 1 -- 172.21.15.106:0/3919102310 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f3cc4059990 con 0x7f3cd80cd650 2024-04-26T21:48:19.907 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 msgr2=0x7f3cb806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 0x7f3cb806ef60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8034000 tx=0x7f3cc802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 msgr2=0x7f3cd814b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c7f0 tx=0x7f3cc400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 shutdown_connections 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cd815cdf0 0x7f3cd809e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cd80cd650 0x7f3cd814b670 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3cb806caa0 0x7f3cb806ef60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 --2- 172.21.15.106:0/3919102310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3cd80b4f50 0x7f3cd814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 >> 172.21.15.106:0/3919102310 conn(0x7f3cd80b2d50 msgr2=0x7f3cd80d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 shutdown_connections 2024-04-26T21:48:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:19.909+0000 7f3cdd000640 1 -- 172.21.15.106:0/3919102310 wait complete. 2024-04-26T21:48:19.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-26T21:48:19.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:19.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:20.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[50725]: fuse finished with error 0 and tester_r 0 2024-04-26T21:48:20.008 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:26.110 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:48:26.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:26.110 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:48:26.169 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:48:26.170 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:26.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:26.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.441+0000 7f936fe00640 1 Processor -- start 2024-04-26T21:48:26.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.441+0000 7f936fe00640 1 -- start start 2024-04-26T21:48:26.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.442+0000 7f936fe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9370145570 0x7f93700d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.442+0000 7f936fe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9370073110 con 0x7f9370073ed0 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.442+0000 7f936fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93700733f0 con 0x7f9370160050 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.442+0000 7f936fe00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f93701606b0 con 0x7f9370145570 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f936ea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9370145570 0x7f93700d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f9367e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9370073ed0 0x7f937015f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56696/0 (socket says 172.21.15.106:56696) 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f9367e00640 1 -- 172.21.15.106:0/2003174629 learned_addr learned my addr 172.21.15.106:0/2003174629 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:26.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f936ea00640 1 -- 172.21.15.106:0/2003174629 >> v1:172.21.15.204:6789/0 conn(0x7f9370160050 legacy=0x7f93700cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:26.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f936ea00640 1 -- 172.21.15.106:0/2003174629 >> v1:172.21.15.106:6789/0 conn(0x7f9370073ed0 legacy=0x7f937015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f936ea00640 1 -- 172.21.15.106:0/2003174629 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93700736d0 con 0x7f9370145570 2024-04-26T21:48:26.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.443+0000 7f936e000640 1 -- 172.21.15.106:0/2003174629 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2978218225 0 0) 0x7f9370073110 con 0x7f9370073ed0 2024-04-26T21:48:26.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.444+0000 7f936ea00640 1 --2- 172.21.15.106:0/2003174629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370145570 0x7f93700d0e70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f93600097a0 tx=0x7f936002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ab29494f9d5242 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936e000640 1 -- 172.21.15.106:0/2003174629 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360037070 con 0x7f9370145570 2024-04-26T21:48:26.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936e000640 1 -- 172.21.15.106:0/2003174629 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936002ae50 con 0x7f9370145570 2024-04-26T21:48:26.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936e000640 1 -- 172.21.15.106:0/2003174629 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360032660 con 0x7f9370145570 2024-04-26T21:48:26.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 -- 172.21.15.106:0/2003174629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370145570 msgr2=0x7f93700d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 --2- 172.21.15.106:0/2003174629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370145570 0x7f93700d0e70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f93600097a0 tx=0x7f936002a140 comp rx=0 tx=0).stop 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 -- 172.21.15.106:0/2003174629 shutdown_connections 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 --2- 172.21.15.106:0/2003174629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370145570 0x7f93700d0e70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 -- 172.21.15.106:0/2003174629 >> 172.21.15.106:0/2003174629 conn(0x7f937006c740 msgr2=0x7f937006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 -- 172.21.15.106:0/2003174629 shutdown_connections 2024-04-26T21:48:26.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.445+0000 7f936fe00640 1 -- 172.21.15.106:0/2003174629 wait complete. 2024-04-26T21:48:26.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.447+0000 7f936fe00640 1 Processor -- start 2024-04-26T21:48:26.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.447+0000 7f936fe00640 1 -- start start 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370073ed0 0x7f937013a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370145570 0x7f9370142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93700d13b0 con 0x7f9370160050 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9370145950 con 0x7f9370073ed0 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.448+0000 7f936fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93700d1530 con 0x7f9370145570 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48588/0 (socket says 172.21.15.106:48588) 2024-04-26T21:48:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 -- 172.21.15.106:0/2090094306 learned_addr learned my addr 172.21.15.106:0/2090094306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f9367e00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370145570 0x7f9370142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936ea00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370073ed0 0x7f937013a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 -- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370145570 msgr2=0x7f9370142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370145570 0x7f9370142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 -- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370073ed0 msgr2=0x7f937013a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370073ed0 0x7f937013a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 -- 172.21.15.106:0/2090094306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93701f8780 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.449+0000 7f936f400640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f936800b990 tx=0x7f936800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.450+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368019070 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.450+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936800cba0 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.450+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93701f8a60 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.450+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368014440 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.450+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93701f9020 con 0x7f9370160050 2024-04-26T21:48:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.451+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93700b5980 con 0x7f9370160050 2024-04-26T21:48:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.452+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f936801c070 con 0x7f9370160050 2024-04-26T21:48:26.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.453+0000 7f9366a00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 0x7f934c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.453+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f936808bc50 con 0x7f9370160050 2024-04-26T21:48:26.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.453+0000 7f936ea00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 0x7f934c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.454+0000 7f936ea00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 0x7f934c06ef90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9360034000 tx=0x7f936002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.458+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93700b5980 con 0x7f9370160050 2024-04-26T21:48:26.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.658+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93701f9220 con 0x7f9370160050 2024-04-26T21:48:26.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.661+0000 7f9366a00640 1 -- 172.21.15.106:0/2090094306 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16625 (secure 0 0 0) 0x7f93701f9220 con 0x7f9370160050 2024-04-26T21:48:26.662 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:26.662 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":45,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:03.337384+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.663+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 msgr2=0x7f934c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.663+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 0x7f934c06ef90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9360034000 tx=0x7f936002e000 comp rx=0 tx=0).stop 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.663+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 msgr2=0x7f9370143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.663+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f936800b990 tx=0x7f936800be50 comp rx=0 tx=0).stop 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 shutdown_connections 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370160050 0x7f9370143470 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f934c06cad0 0x7f934c06ef90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370145570 0x7f9370142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 --2- 172.21.15.106:0/2090094306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370073ed0 0x7f937013a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 >> 172.21.15.106:0/2090094306 conn(0x7f937006c740 msgr2=0x7f93700cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:26.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 shutdown_connections 2024-04-26T21:48:26.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.664+0000 7f936fe00640 1 -- 172.21.15.106:0/2090094306 wait complete. 2024-04-26T21:48:26.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:59.554855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:48:00.766548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:26.684 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:26.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.883+0000 7f5748600640 1 Processor -- start 2024-04-26T21:48:26.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.883+0000 7f5748600640 1 -- start start 2024-04-26T21:48:26.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.884+0000 7f5748600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f574006ce80 0x7f57400b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.884+0000 7f5748600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57400af800 con 0x7f574015d100 2024-04-26T21:48:26.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.884+0000 7f5748600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57400ae340 con 0x7f57400b5ca0 2024-04-26T21:48:26.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.884+0000 7f5748600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f57400afaa0 con 0x7f574006ce80 2024-04-26T21:48:26.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f574006ce80 0x7f57400b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f574006ce80 0x7f57400b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48602/0 (socket says 172.21.15.106:48602) 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 -- 172.21.15.106:0/2767091409 learned_addr learned my addr 172.21.15.106:0/2767091409 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 -- 172.21.15.106:0/2767091409 >> v1:172.21.15.204:6789/0 conn(0x7f57400b5ca0 legacy=0x7f57400b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 -- 172.21.15.106:0/2767091409 >> v1:172.21.15.106:6789/0 conn(0x7f574015d100 legacy=0x7f574006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f5745200640 1 -- 172.21.15.106:0/2767091409 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57400afd80 con 0x7f574006ce80 2024-04-26T21:48:26.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.885+0000 7f573fe00640 1 -- 172.21.15.106:0/2767091409 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387682 0 0) 0x7f57400af800 con 0x7f574015d100 2024-04-26T21:48:26.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.886+0000 7f5745200640 1 --2- 172.21.15.106:0/2767091409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574006ce80 0x7f57400b3c30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97780aec364c94c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.887+0000 7f573fe00640 1 -- 172.21.15.106:0/2767091409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c037070 con 0x7f574006ce80 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.887+0000 7f573fe00640 1 -- 172.21.15.106:0/2767091409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c032440 con 0x7f574006ce80 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.887+0000 7f573fe00640 1 -- 172.21.15.106:0/2767091409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c032810 con 0x7f574006ce80 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.887+0000 7f5748600640 1 -- 172.21.15.106:0/2767091409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574006ce80 msgr2=0x7f57400b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.887+0000 7f5748600640 1 --2- 172.21.15.106:0/2767091409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574006ce80 0x7f57400b3c30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c02a210 comp rx=0 tx=0).stop 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.888+0000 7f5748600640 1 -- 172.21.15.106:0/2767091409 shutdown_connections 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.888+0000 7f5748600640 1 --2- 172.21.15.106:0/2767091409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574006ce80 0x7f57400b3c30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.888+0000 7f5748600640 1 -- 172.21.15.106:0/2767091409 >> 172.21.15.106:0/2767091409 conn(0x7f57400b2d50 msgr2=0x7f57400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.888+0000 7f5748600640 1 -- 172.21.15.106:0/2767091409 shutdown_connections 2024-04-26T21:48:26.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.888+0000 7f5748600640 1 -- 172.21.15.106:0/2767091409 wait complete. 2024-04-26T21:48:26.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.889+0000 7f5748600640 1 Processor -- start 2024-04-26T21:48:26.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 -- start start 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f574006ce80 0x7f574015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f574015d100 0x7f57401f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f574009ee50 con 0x7f57400b5ca0 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b0340 con 0x7f574015d100 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.890+0000 7f5748600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5740160b70 con 0x7f574006ce80 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48610/0 (socket says 172.21.15.106:48610) 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 -- 172.21.15.106:0/3330464346 learned_addr learned my addr 172.21.15.106:0/3330464346 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 -- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f574006ce80 msgr2=0x7f574015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f574006ce80 0x7f574015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 -- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f574015d100 msgr2=0x7f57401f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f574015d100 0x7f57401f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.891+0000 7f5745200640 1 -- 172.21.15.106:0/3330464346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57401f47f0 con 0x7f57400b5ca0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f5745200640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c037040 con 0x7f57400b5ca0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c004250 con 0x7f57400b5ca0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c03c670 con 0x7f57400b5ca0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57401f4ad0 con 0x7f57400b5ca0 2024-04-26T21:48:26.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.892+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401f5090 con 0x7f57400b5ca0 2024-04-26T21:48:26.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.893+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5740069650 con 0x7f57400b5ca0 2024-04-26T21:48:26.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.894+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f572c03a070 con 0x7f57400b5ca0 2024-04-26T21:48:26.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.896+0000 7f573d600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 0x7f572006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:26.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.896+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f572c0ac830 con 0x7f57400b5ca0 2024-04-26T21:48:26.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.896+0000 7f5745c00640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 0x7f572006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:26.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.897+0000 7f5745c00640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 0x7f572006ef90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5734001560 tx=0x7f5734007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:26.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:26.900+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f572c041020 con 0x7f57400b5ca0 2024-04-26T21:48:27.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.098+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5740073d00 con 0x7f57400b5ca0 2024-04-26T21:48:27.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.099+0000 7f573d600640 1 -- 172.21.15.106:0/3330464346 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f572c07a9d0 con 0x7f57400b5ca0 2024-04-26T21:48:27.100 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 msgr2=0x7f572006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 0x7f572006ef90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5734001560 tx=0x7f5734007930 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 msgr2=0x7f57401f5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 shutdown_connections 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f574015d100 0x7f57401f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.101+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57400b5ca0 0x7f57401f5890 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.102+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f572006cad0 0x7f572006ef90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.102+0000 7f5748600640 1 --2- 172.21.15.106:0/3330464346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f574006ce80 0x7f574015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.102+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 >> 172.21.15.106:0/3330464346 conn(0x7f57400b2d50 msgr2=0x7f57400b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:27.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.102+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 shutdown_connections 2024-04-26T21:48:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.102+0000 7f5748600640 1 -- 172.21.15.106:0/3330464346 wait complete. 2024-04-26T21:48:27.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:27.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.324+0000 7fed0b800640 1 Processor -- start 2024-04-26T21:48:27.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.324+0000 7fed0b800640 1 -- start start 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.325+0000 7fed0b800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed040d19c0 0x7fed040d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.325+0000 7fed0b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed040d1da0 0x7fed04162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.325+0000 7fed0b800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed040d3830 con 0x7fed040b6170 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.325+0000 7fed0b800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fed040d40d0 con 0x7fed040d19c0 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.325+0000 7fed0b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fed040d3b10 con 0x7fed040d1da0 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed040d19c0 0x7fed040d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed040d19c0 0x7fed040d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48614/0 (socket says 172.21.15.106:48614) 2024-04-26T21:48:27.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 -- 172.21.15.106:0/1348015145 learned_addr learned my addr 172.21.15.106:0/1348015145 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 -- 172.21.15.106:0/1348015145 >> v2:172.21.15.204:3300/0 conn(0x7fed040d1da0 msgr2=0x7fed04162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 --2- 172.21.15.106:0/1348015145 >> v2:172.21.15.204:3300/0 conn(0x7fed040d1da0 0x7fed04162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 -- 172.21.15.106:0/1348015145 >> v1:172.21.15.106:6789/0 conn(0x7fed040b6170 legacy=0x7fed040c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.326+0000 7fed08e00640 1 -- 172.21.15.106:0/1348015145 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed040d3df0 con 0x7fed040d19c0 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.327+0000 7fed08e00640 1 --2- 172.21.15.106:0/1348015145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d19c0 0x7fed040d2f00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed0408e590 tx=0x7fecf402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7d6ef34a2c726 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed03400640 1 -- 172.21.15.106:0/1348015145 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4037070 con 0x7fed040d19c0 2024-04-26T21:48:27.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed03400640 1 -- 172.21.15.106:0/1348015145 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf4032440 con 0x7fed040d19c0 2024-04-26T21:48:27.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed03400640 1 -- 172.21.15.106:0/1348015145 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4032810 con 0x7fed040d19c0 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 -- 172.21.15.106:0/1348015145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d19c0 msgr2=0x7fed040d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 --2- 172.21.15.106:0/1348015145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d19c0 0x7fed040d2f00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed0408e590 tx=0x7fecf402a1f0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 -- 172.21.15.106:0/1348015145 shutdown_connections 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 --2- 172.21.15.106:0/1348015145 >> v2:172.21.15.204:3300/0 conn(0x7fed040d1da0 0x7fed04162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 --2- 172.21.15.106:0/1348015145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d19c0 0x7fed040d2f00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 -- 172.21.15.106:0/1348015145 >> 172.21.15.106:0/1348015145 conn(0x7fed0409e460 msgr2=0x7fed040d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 -- 172.21.15.106:0/1348015145 shutdown_connections 2024-04-26T21:48:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.328+0000 7fed0b800640 1 -- 172.21.15.106:0/1348015145 wait complete. 2024-04-26T21:48:27.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.330+0000 7fed0b800640 1 Processor -- start 2024-04-26T21:48:27.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.330+0000 7fed0b800640 1 -- start start 2024-04-26T21:48:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 0x7fed04155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 0x7fed041592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed04162b80 con 0x7fed040d1da0 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed040cfc30 con 0x7fed040d19c0 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed0b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed04162d00 con 0x7fed040b6170 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed09800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.331+0000 7fed09800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48630/0 (socket says 172.21.15.106:48630) 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 -- 172.21.15.106:0/4287471685 learned_addr learned my addr 172.21.15.106:0/4287471685 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed08e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 0x7fed04155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 -- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 msgr2=0x7fed04155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed03e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 0x7fed041592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 0x7fed04155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 -- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 msgr2=0x7fed041592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 0x7fed041592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 -- 172.21.15.106:0/4287471685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041f59b0 con 0x7fed040d1da0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed08e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 0x7fed04155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed03e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 0x7fed041592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed09800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b970 tx=0x7fecfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc019070 con 0x7fed040d1da0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.332+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc00cb00 con 0x7fed040d1da0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.333+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc014440 con 0x7fed040d1da0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.333+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041f5c90 con 0x7fed040d1da0 2024-04-26T21:48:27.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.333+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041f6250 con 0x7fed040d1da0 2024-04-26T21:48:27.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.334+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04069650 con 0x7fed040d1da0 2024-04-26T21:48:27.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.335+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fecfc01c070 con 0x7fed040d1da0 2024-04-26T21:48:27.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.336+0000 7fed00c00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 0x7fece406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.336+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fecfc08bc20 con 0x7fed040d1da0 2024-04-26T21:48:27.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.336+0000 7fed08e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 0x7fece406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.337+0000 7fed08e00640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 0x7fece406ef60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fecf4034000 tx=0x7fecf402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.341+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecfc059d30 con 0x7fed040d1da0 2024-04-26T21:48:27.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.539+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fed040d26d0 con 0x7fed040d1da0 2024-04-26T21:48:27.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.540+0000 7fed00c00640 1 -- 172.21.15.106:0/4287471685 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7fecfc059630 con 0x7fed040d1da0 2024-04-26T21:48:27.541 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:48:27.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 msgr2=0x7fece406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 0x7fece406ef60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fecf4034000 tx=0x7fecf402e000 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 msgr2=0x7fed040cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b970 tx=0x7fecfc00be30 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 shutdown_connections 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed040d1da0 0x7fed040cd6a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed040d19c0 0x7fed041592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece406caa0 0x7fece406ef60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 --2- 172.21.15.106:0/4287471685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed040b6170 0x7fed04155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 >> 172.21.15.106:0/4287471685 conn(0x7fed0409e460 msgr2=0x7fed0406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 shutdown_connections 2024-04-26T21:48:27.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.543+0000 7fed0b800640 1 -- 172.21.15.106:0/4287471685 wait complete. 2024-04-26T21:48:27.564 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.759+0000 7f3f39a00640 1 Processor -- start 2024-04-26T21:48:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.759+0000 7f3f39a00640 1 -- start start 2024-04-26T21:48:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f3415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39a00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f340d1470 0x7f3f340b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39a00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3415e3e0 con 0x7f3f340b5ca0 2024-04-26T21:48:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39a00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3f34146b80 con 0x7f3f340d1470 2024-04-26T21:48:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f34161730 con 0x7f3f341467a0 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f39000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f340d1470 0x7f3f340b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.761+0000 7f3f2b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f340b5ca0 0x7f3f340d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56724/0 (socket says 172.21.15.106:56724) 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f2b400640 1 -- 172.21.15.106:0/405138890 learned_addr learned my addr 172.21.15.106:0/405138890 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f33e00640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f3415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f39000640 1 -- 172.21.15.106:0/405138890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 msgr2=0x7f3f3415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f39000640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f3415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f39000640 1 -- 172.21.15.106:0/405138890 >> v1:172.21.15.106:6789/0 conn(0x7f3f340b5ca0 legacy=0x7f3f340d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f39000640 1 -- 172.21.15.106:0/405138890 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f34160b20 con 0x7f3f340d1470 2024-04-26T21:48:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.762+0000 7f3f33e00640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f3415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.763+0000 7f3f39000640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f340b40f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009d90 tx=0x7f3f2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c9bee6f501b9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.763+0000 7f3f33400640 1 -- 172.21.15.106:0/405138890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007740 con 0x7f3f340d1470 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.763+0000 7f3f33400640 1 -- 172.21.15.106:0/405138890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0078e0 con 0x7f3f340d1470 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.763+0000 7f3f33400640 1 -- 172.21.15.106:0/405138890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007cb0 con 0x7f3f340d1470 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 -- 172.21.15.106:0/405138890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 msgr2=0x7f3f340b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f340b40f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009d90 tx=0x7f3f2c02a660 comp rx=0 tx=0).stop 2024-04-26T21:48:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 -- 172.21.15.106:0/405138890 shutdown_connections 2024-04-26T21:48:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f340b40f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 --2- 172.21.15.106:0/405138890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f3415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 -- 172.21.15.106:0/405138890 >> 172.21.15.106:0/405138890 conn(0x7f3f3406c530 msgr2=0x7f3f3406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 -- 172.21.15.106:0/405138890 shutdown_connections 2024-04-26T21:48:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.764+0000 7f3f39a00640 1 -- 172.21.15.106:0/405138890 wait complete. 2024-04-26T21:48:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.766+0000 7f3f39a00640 1 Processor -- start 2024-04-26T21:48:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.766+0000 7f3f39a00640 1 -- start start 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f340b5ca0 0x7f3f3415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f34144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f341613c0 con 0x7f3f340d1470 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f341610e0 con 0x7f3f340b5ca0 2024-04-26T21:48:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f39a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f340cf100 con 0x7f3f341467a0 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f2b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f2b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48636/0 (socket says 172.21.15.106:48636) 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.767+0000 7f3f2b400640 1 -- 172.21.15.106:0/1576599848 learned_addr learned my addr 172.21.15.106:0/1576599848 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f39000640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f34144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f33e00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f340b5ca0 0x7f3f3415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 -- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 msgr2=0x7f3f34144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f34144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 -- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f340b5ca0 msgr2=0x7f3f3415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f340b5ca0 0x7f3f3415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 -- 172.21.15.106:0/1576599848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3415c7f0 con 0x7f3f340d1470 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f2b400640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c400 tx=0x7f3f1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c017070 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1c005ce0 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0072f0 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f34144dd0 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.768+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3415cd70 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.770+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f340693c0 con 0x7f3f340d1470 2024-04-26T21:48:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.771+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3f1c01a070 con 0x7f3f340d1470 2024-04-26T21:48:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.772+0000 7f3f30c00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 0x7f3f1006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.772+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3f1c08b250 con 0x7f3f340d1470 2024-04-26T21:48:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.772+0000 7f3f33e00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 0x7f3f1006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.773+0000 7f3f33e00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 0x7f3f1006ef60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b840 tx=0x7f3f24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:27.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.776+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f3415e3e0 con 0x7f3f340d1470 2024-04-26T21:48:27.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.974+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f3415dab0 con 0x7f3f340d1470 2024-04-26T21:48:27.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.975+0000 7f3f30c00640 1 -- 172.21.15.106:0/1576599848 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f3f1c059400 con 0x7f3f340d1470 2024-04-26T21:48:27.976 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:48:27.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 msgr2=0x7f3f1006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 0x7f3f1006ef60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b840 tx=0x7f3f24008040 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 msgr2=0x7f3f341f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c400 tx=0x7f3f1c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 shutdown_connections 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3f1006caa0 0x7f3f1006ef60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f341467a0 0x7f3f34144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f340d1470 0x7f3f341f4060 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 --2- 172.21.15.106:0/1576599848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f340b5ca0 0x7f3f3415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.978+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 >> 172.21.15.106:0/1576599848 conn(0x7f3f3406c530 msgr2=0x7f3f34141900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.979+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 shutdown_connections 2024-04-26T21:48:27.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:27.979+0000 7f3f39a00640 1 -- 172.21.15.106:0/1576599848 wait complete. 2024-04-26T21:48:27.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:28.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.189+0000 7fccd7200640 1 Processor -- start 2024-04-26T21:48:28.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.189+0000 7fccd7200640 1 -- start start 2024-04-26T21:48:28.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fccd815f1b0 0x7fccd80d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fccd8162b00 0x7fccd8162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd8163480 0x7fccd80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fccd8163910 con 0x7fccd815f1b0 2024-04-26T21:48:28.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fccd80d3f50 con 0x7fccd8162b00 2024-04-26T21:48:28.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccd80d36b0 con 0x7fccd8163480 2024-04-26T21:48:28.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fccd815f1b0 0x7fccd80d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fccd815f1b0 0x7fccd80d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56946/0 (socket says 172.21.15.106:56946) 2024-04-26T21:48:28.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.191+0000 7fccd5e00640 1 -- 172.21.15.106:0/4138683456 learned_addr learned my addr 172.21.15.106:0/4138683456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:28.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5400640 1 --2- 172.21.15.106:0/4138683456 >> v2:172.21.15.204:3300/0 conn(0x7fccd8162b00 0x7fccd8162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd6800640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd8163480 0x7fccd80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5e00640 1 -- 172.21.15.106:0/4138683456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd8163480 msgr2=0x7fccd80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5e00640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd8163480 0x7fccd80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5e00640 1 -- 172.21.15.106:0/4138683456 >> v2:172.21.15.204:3300/0 conn(0x7fccd8162b00 msgr2=0x7fccd8162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5e00640 1 --2- 172.21.15.106:0/4138683456 >> v2:172.21.15.204:3300/0 conn(0x7fccd8162b00 0x7fccd8162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.192+0000 7fccd5e00640 1 -- 172.21.15.106:0/4138683456 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd80d3990 con 0x7fccd815f1b0 2024-04-26T21:48:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.193+0000 7fccd5e00640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd815f1b0 0x7fccd80d5990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fccd808df60 tx=0x7fccc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3848113925e05a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.193+0000 7fccd4a00640 1 -- 172.21.15.106:0/4138683456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8037070 con 0x7fccd815f1b0 2024-04-26T21:48:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.193+0000 7fccd4a00640 1 -- 172.21.15.106:0/4138683456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc8032440 con 0x7fccd815f1b0 2024-04-26T21:48:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.193+0000 7fccd4a00640 1 -- 172.21.15.106:0/4138683456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8032810 con 0x7fccd815f1b0 2024-04-26T21:48:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 -- 172.21.15.106:0/4138683456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd815f1b0 msgr2=0x7fccd80d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd815f1b0 0x7fccd80d5990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fccd808df60 tx=0x7fccc802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 -- 172.21.15.106:0/4138683456 shutdown_connections 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd8163480 0x7fccd80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 --2- 172.21.15.106:0/4138683456 >> v2:172.21.15.204:3300/0 conn(0x7fccd8162b00 0x7fccd8162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 --2- 172.21.15.106:0/4138683456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd815f1b0 0x7fccd80d5990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 -- 172.21.15.106:0/4138683456 >> 172.21.15.106:0/4138683456 conn(0x7fccd809e230 msgr2=0x7fccd80d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 -- 172.21.15.106:0/4138683456 shutdown_connections 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.194+0000 7fccd7200640 1 -- 172.21.15.106:0/4138683456 wait complete. 2024-04-26T21:48:28.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.196+0000 7fccd7200640 1 Processor -- start 2024-04-26T21:48:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.196+0000 7fccd7200640 1 -- start start 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 0x7fccd8156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8163480 0x7fccd80ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fccd80d33a0 con 0x7fccd8162b00 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd80b7d90 con 0x7fccd8163480 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccd80d0010 con 0x7fccd815f1b0 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56950/0 (socket says 172.21.15.106:56950) 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.197+0000 7fccd5400640 1 -- 172.21.15.106:0/2409213384 learned_addr learned my addr 172.21.15.106:0/2409213384 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd6800640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8163480 0x7fccd80ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5e00640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 0x7fccd8156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 -- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 msgr2=0x7fccd8156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 0x7fccd8156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 -- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8163480 msgr2=0x7fccd80ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8163480 0x7fccd80ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 -- 172.21.15.106:0/2409213384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81f5200 con 0x7fccd8162b00 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5e00640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 0x7fccd8156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd5400640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fccc0007e60 tx=0x7fccc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0014620 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc0019070 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd81f54e0 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.198+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0014a00 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.199+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81f5aa0 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.200+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd80693c0 con 0x7fccd8162b00 2024-04-26T21:48:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.201+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fccc0014ba0 con 0x7fccd8162b00 2024-04-26T21:48:28.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.202+0000 7fccce000640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 0x7fccb406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.202+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fccc008c7e0 con 0x7fccd8162b00 2024-04-26T21:48:28.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.202+0000 7fccd5e00640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 0x7fccb406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.203+0000 7fccd5e00640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 0x7fccb406ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.206+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccc005a8f0 con 0x7fccd8162b00 2024-04-26T21:48:28.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.414+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fccd80cad00 con 0x7fccd8162b00 2024-04-26T21:48:28.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.415+0000 7fccce000640 1 -- 172.21.15.106:0/2409213384 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fccc005a1f0 con 0x7fccd8162b00 2024-04-26T21:48:28.416 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.417+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 msgr2=0x7fccb406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.417+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 0x7fccb406ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.417+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 msgr2=0x7fccd80ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.417+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fccc0007e60 tx=0x7fccc000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 shutdown_connections 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8163480 0x7fccd80ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccd8162b00 0x7fccd80ca620 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fccb406caa0 0x7fccb406ef60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 --2- 172.21.15.106:0/2409213384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd815f1b0 0x7fccd8156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 >> 172.21.15.106:0/2409213384 conn(0x7fccd809e230 msgr2=0x7fccd80b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 shutdown_connections 2024-04-26T21:48:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.418+0000 7fccd7200640 1 -- 172.21.15.106:0/2409213384 wait complete. 2024-04-26T21:48:28.435 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:48:28.435 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:28.492 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:28.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.683+0000 7f0b32200640 1 Processor -- start 2024-04-26T21:48:28.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.684+0000 7f0b32200640 1 -- start start 2024-04-26T21:48:28.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.685+0000 7f0b32200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0b2c0d0e70 0x7f0b2c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.685+0000 7f0b32200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2c146a60 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.685+0000 7f0b32200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2c06dd60 con 0x7f0b2c146680 2024-04-26T21:48:28.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.685+0000 7f0b32200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2c06e970 con 0x7f0b2c0d0e70 2024-04-26T21:48:28.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b31800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b2c0b3b20 0x7f0b2c0b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43424/0 (socket says 172.21.15.106:43424) 2024-04-26T21:48:28.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b31800640 1 -- 172.21.15.106:0/3426377060 learned_addr learned my addr 172.21.15.106:0/3426377060 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306467615 0 0) 0x7f0b2c146a60 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b14003520 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2be00640 1 --2- 172.21.15.106:0/3426377060 >> v2:172.21.15.204:3300/0 conn(0x7f0b2c0d0e70 0x7f0b2c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 496255755 0 0) 0x7f0b14003520 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 >> v2:172.21.15.204:3300/0 conn(0x7f0b2c0d0e70 msgr2=0x7f0b2c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 --2- 172.21.15.106:0/3426377060 >> v2:172.21.15.204:3300/0 conn(0x7f0b2c0d0e70 0x7f0b2c0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 >> v1:172.21.15.204:6789/0 conn(0x7f0b2c146680 legacy=0x7f0b2c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c06e040 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f0b240019f0 con 0x7f0b2c0b3b20 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.686+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 >> v1:172.21.15.106:6789/0 conn(0x7f0b2c0b3b20 legacy=0x7f0b2c0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b140039b0 0x7f0b14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 0x7f0b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b140051c0 0x7f0b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1400c0e0 con 0x7f0b14004800 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1400c290 con 0x7f0b140039b0 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1400c490 con 0x7f0b140051c0 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 0x7f0b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b30e00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b140039b0 0x7f0b14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 -- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b140051c0 msgr2=0x7f0b1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b140051c0 0x7f0b1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 -- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b140039b0 msgr2=0x7f0b14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b140039b0 0x7f0b14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b2be00640 1 -- 172.21.15.106:0/3426377060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b14003520 con 0x7f0b14004800 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b30e00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b140039b0 0x7f0b14003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.687+0000 7f0b31800640 1 -- 172.21.15.106:0/3426377060 reap_dead start 2024-04-26T21:48:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.688+0000 7f0b2be00640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 0x7f0b14004c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800ba30 tx=0x7f0b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efe73c38accdf3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030070 con 0x7f0b14004800 2024-04-26T21:48:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1802ade0 con 0x7f0b14004800 2024-04-26T21:48:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b2b400640 1 -- 172.21.15.106:0/3426377060 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800b600 con 0x7f0b14004800 2024-04-26T21:48:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 -- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 msgr2=0x7f0b14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 0x7f0b14004c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800ba30 tx=0x7f0b14005830 comp rx=0 tx=0).stop 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 -- 172.21.15.106:0/3426377060 shutdown_connections 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 --2- 172.21.15.106:0/3426377060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b14004800 0x7f0b14004c80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 -- 172.21.15.106:0/3426377060 >> 172.21.15.106:0/3426377060 conn(0x7f0b2c06c740 msgr2=0x7f0b2c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 -- 172.21.15.106:0/3426377060 shutdown_connections 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.689+0000 7f0b32200640 1 -- 172.21.15.106:0/3426377060 wait complete. 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.691+0000 7f0b32200640 1 Processor -- start 2024-04-26T21:48:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.691+0000 7f0b32200640 1 -- start start 2024-04-26T21:48:28.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 0x7f0b2c13b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 0x7f0b2c15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c0cd6a0 con 0x7f0b2c0cc390 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c06e600 con 0x7f0b14004800 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b32200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2c0cd820 con 0x7f0b2c0cbf10 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.692+0000 7f0b31800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56968/0 (socket says 172.21.15.106:56968) 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 -- 172.21.15.106:0/774273962 learned_addr learned my addr 172.21.15.106:0/774273962 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b30e00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 0x7f0b2c13b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b2be00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 0x7f0b2c15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 -- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 msgr2=0x7f0b2c15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 0x7f0b2c15bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 -- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 msgr2=0x7f0b2c13b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 0x7f0b2c13b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 -- 172.21.15.106:0/774273962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c13bac0 con 0x7f0b2c0cc390 2024-04-26T21:48:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b30e00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 0x7f0b2c13b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b2be00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 0x7f0b2c15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b31800640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000f80 tx=0x7f0b2400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400a5b0 con 0x7f0b2c0cc390 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b2400a770 con 0x7f0b2c0cc390 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.693+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400ab60 con 0x7f0b2c0cc390 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.694+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c13bd80 con 0x7f0b2c0cc390 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.694+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c13c340 con 0x7f0b2c0cc390 2024-04-26T21:48:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.695+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c0693c0 con 0x7f0b2c0cc390 2024-04-26T21:48:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.696+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0b24011070 con 0x7f0b2c0cc390 2024-04-26T21:48:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.696+0000 7f0b28c00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 0x7f0b0c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.697+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0b2408b500 con 0x7f0b2c0cc390 2024-04-26T21:48:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.697+0000 7f0b30e00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 0x7f0b0c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.697+0000 7f0b30e00640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 0x7f0b0c06ef60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0cd2f0 tx=0x7f0b20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.701+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b24059610 con 0x7f0b2c0cc390 2024-04-26T21:48:28.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.887+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b2c0b43b0 con 0x7f0b2c0cc390 2024-04-26T21:48:28.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.890+0000 7f0b28c00640 1 -- 172.21.15.106:0/774273962 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16625 (secure 0 0 0) 0x7f0b24058f10 con 0x7f0b2c0cc390 2024-04-26T21:48:28.891 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:28.891 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":45,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:03.337384+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.892+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 msgr2=0x7f0b0c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.892+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 0x7f0b0c06ef60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0cd2f0 tx=0x7f0b20009040 comp rx=0 tx=0).stop 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.892+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 msgr2=0x7f0b2c15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.892+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000f80 tx=0x7f0b2400cf00 comp rx=0 tx=0).stop 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 shutdown_connections 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2c0cc390 0x7f0b2c15c0f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0b0c06caa0 0x7f0b0c06ef60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b2c0cbf10 0x7f0b2c15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 --2- 172.21.15.106:0/774273962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b14004800 0x7f0b2c13b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:28.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 >> 172.21.15.106:0/774273962 conn(0x7f0b2c06c740 msgr2=0x7f0b2c06cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:28.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 shutdown_connections 2024-04-26T21:48:28.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:28.893+0000 7f0b32200640 1 -- 172.21.15.106:0/774273962 wait complete. 2024-04-26T21:48:28.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:59.554855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:48:00.766548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:28.914 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:29.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.104+0000 7f958d400640 1 Processor -- start 2024-04-26T21:48:29.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.104+0000 7f958d400640 1 -- start start 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f958d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 0x7f958806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f958d400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95880add80 0x7f958806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f958d400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95880af800 con 0x7f95880b5ca0 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f958d400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f95880ae340 con 0x7f95880add80 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f958d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f95880afaa0 con 0x7f958815d100 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f9587200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95880add80 0x7f958806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f9587200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95880add80 0x7f958806ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56984/0 (socket says 172.21.15.106:56984) 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f9587200640 1 -- 172.21.15.106:0/489356189 learned_addr learned my addr 172.21.15.106:0/489356189 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.106+0000 7f957de00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f95880b5ca0 0x7f9588072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43426/0 (socket says 172.21.15.106:43426) 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9586800640 1 --2- 172.21.15.106:0/489356189 >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 0x7f958806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9587200640 1 -- 172.21.15.106:0/489356189 >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 msgr2=0x7f958806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9587200640 1 --2- 172.21.15.106:0/489356189 >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 0x7f958806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9587200640 1 -- 172.21.15.106:0/489356189 >> v1:172.21.15.106:6789/0 conn(0x7f95880b5ca0 legacy=0x7f9588072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9587200640 1 -- 172.21.15.106:0/489356189 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95880afd80 con 0x7f95880add80 2024-04-26T21:48:29.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.107+0000 7f9586800640 1 --2- 172.21.15.106:0/489356189 >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 0x7f958806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:29.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.108+0000 7f9587200640 1 --2- 172.21.15.106:0/489356189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f958806ccb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f95880730d0 tx=0x7f958002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1765f8a1f4b421 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.108+0000 7f9585e00640 1 -- 172.21.15.106:0/489356189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580036070 con 0x7f95880add80 2024-04-26T21:48:29.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.108+0000 7f9585e00640 1 -- 172.21.15.106:0/489356189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9580031600 con 0x7f95880add80 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.108+0000 7f9585e00640 1 -- 172.21.15.106:0/489356189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800319d0 con 0x7f95880add80 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 -- 172.21.15.106:0/489356189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 msgr2=0x7f958806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 --2- 172.21.15.106:0/489356189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f958806ccb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f95880730d0 tx=0x7f958002a730 comp rx=0 tx=0).stop 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 -- 172.21.15.106:0/489356189 shutdown_connections 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 --2- 172.21.15.106:0/489356189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f958806ccb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 --2- 172.21.15.106:0/489356189 >> v2:172.21.15.204:3300/0 conn(0x7f958815d100 0x7f958806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 -- 172.21.15.106:0/489356189 >> 172.21.15.106:0/489356189 conn(0x7f95880b2d50 msgr2=0x7f95880b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 -- 172.21.15.106:0/489356189 shutdown_connections 2024-04-26T21:48:29.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.109+0000 7f958d400640 1 -- 172.21.15.106:0/489356189 wait complete. 2024-04-26T21:48:29.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.111+0000 7f958d400640 1 Processor -- start 2024-04-26T21:48:29.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.111+0000 7f958d400640 1 -- start start 2024-04-26T21:48:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880b5ca0 0x7f958815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 0x7f958806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f958809ee50 con 0x7f95880add80 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95880b0340 con 0x7f958815d100 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f958d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9588160b70 con 0x7f95880b5ca0 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f9586800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f9586800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56990/0 (socket says 172.21.15.106:56990) 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f9586800640 1 -- 172.21.15.106:0/599156675 learned_addr learned my addr 172.21.15.106:0/599156675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f9586800640 1 -- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880b5ca0 msgr2=0x7f958815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.112+0000 7f9586800640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880b5ca0 0x7f958815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9586800640 1 -- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 msgr2=0x7f958806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9587200640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 0x7f958806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9586800640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 0x7f958806ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9586800640 1 -- 172.21.15.106:0/599156675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958809c8d0 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9587200640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 0x7f958806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f9586800640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f957800c7f0 tx=0x7f957800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578015070 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9578007820 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958806e190 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578007bd0 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.113+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958806e750 con 0x7f95880add80 2024-04-26T21:48:29.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.115+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9588069650 con 0x7f95880add80 2024-04-26T21:48:29.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.116+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9578018070 con 0x7f95880add80 2024-04-26T21:48:29.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.116+0000 7f957f400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 0x7f956406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.117+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f957808c0c0 con 0x7f95880add80 2024-04-26T21:48:29.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.117+0000 7f957de00640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 0x7f956406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.118+0000 7f957de00640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 0x7f956406ef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9570000f60 tx=0x7f9570007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.121+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f957805a160 con 0x7f95880add80 2024-04-26T21:48:29.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.321+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9588073d00 con 0x7f95880add80 2024-04-26T21:48:29.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.322+0000 7f957f400640 1 -- 172.21.15.106:0/599156675 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f9578059a60 con 0x7f95880add80 2024-04-26T21:48:29.323 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.324+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 msgr2=0x7f956406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 0x7f956406ef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9570000f60 tx=0x7f9570007a00 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 msgr2=0x7f9588147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f957800c7f0 tx=0x7f957800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 shutdown_connections 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f958815d100 0x7f958806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f956406caa0 0x7f956406ef60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880b5ca0 0x7f958815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 --2- 172.21.15.106:0/599156675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95880add80 0x7f9588147510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 >> 172.21.15.106:0/599156675 conn(0x7f95880b2d50 msgr2=0x7f9588155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:29.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 shutdown_connections 2024-04-26T21:48:29.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.325+0000 7f958d400640 1 -- 172.21.15.106:0/599156675 wait complete. 2024-04-26T21:48:29.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:29.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.544+0000 7f9d3d600640 1 Processor -- start 2024-04-26T21:48:29.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.545+0000 7f9d3d600640 1 -- start start 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d3d600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d380b6170 0x7f9d380c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d3d600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 0x7f9d38162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d3d600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d380d3830 con 0x7f9d380d19c0 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d3d600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9d380d40d0 con 0x7f9d380b6170 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d3d600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9d380d3b10 con 0x7f9d380d1da0 2024-04-26T21:48:29.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d36a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9d380d19c0 0x7f9d380d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43430/0 (socket says 172.21.15.106:43430) 2024-04-26T21:48:29.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d36a00640 1 -- 172.21.15.106:0/4056809486 learned_addr learned my addr 172.21.15.106:0/4056809486 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:29.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.546+0000 7f9d36000640 1 --2- 172.21.15.106:0/4056809486 >> v2:172.21.15.106:3300/0 conn(0x7f9d380b6170 0x7f9d380c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d37400640 1 --2- 172.21.15.106:0/4056809486 >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 0x7f9d38162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d36000640 1 -- 172.21.15.106:0/4056809486 >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 msgr2=0x7f9d38162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d36000640 1 --2- 172.21.15.106:0/4056809486 >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 0x7f9d38162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d36000640 1 -- 172.21.15.106:0/4056809486 >> v1:172.21.15.106:6789/0 conn(0x7f9d380d19c0 legacy=0x7f9d380d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d36000640 1 -- 172.21.15.106:0/4056809486 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d380d3df0 con 0x7f9d380b6170 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d37400640 1 --2- 172.21.15.106:0/4056809486 >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 0x7f9d38162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.547+0000 7f9d35600640 1 -- 172.21.15.106:0/4056809486 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3071048977 0 0) 0x7f9d380d3830 con 0x7f9d380d19c0 2024-04-26T21:48:29.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.548+0000 7f9d36000640 1 --2- 172.21.15.106:0/4056809486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380b6170 0x7f9d380c8c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9d3808e590 tx=0x7f9d2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bbf7d97a2c4304 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.548+0000 7f9d35600640 1 -- 172.21.15.106:0/4056809486 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20037070 con 0x7f9d380b6170 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.548+0000 7f9d35600640 1 -- 172.21.15.106:0/4056809486 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d20032440 con 0x7f9d380b6170 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.548+0000 7f9d35600640 1 -- 172.21.15.106:0/4056809486 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2002ae60 con 0x7f9d380b6170 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 -- 172.21.15.106:0/4056809486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380b6170 msgr2=0x7f9d380c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 --2- 172.21.15.106:0/4056809486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380b6170 0x7f9d380c8c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9d3808e590 tx=0x7f9d2002a110 comp rx=0 tx=0).stop 2024-04-26T21:48:29.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 -- 172.21.15.106:0/4056809486 shutdown_connections 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 --2- 172.21.15.106:0/4056809486 >> v2:172.21.15.204:3300/0 conn(0x7f9d380d1da0 0x7f9d38162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 --2- 172.21.15.106:0/4056809486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380b6170 0x7f9d380c8c70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 -- 172.21.15.106:0/4056809486 >> 172.21.15.106:0/4056809486 conn(0x7f9d3809e460 msgr2=0x7f9d380d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 -- 172.21.15.106:0/4056809486 shutdown_connections 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.549+0000 7f9d3d600640 1 -- 172.21.15.106:0/4056809486 wait complete. 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.551+0000 7f9d3d600640 1 Processor -- start 2024-04-26T21:48:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.551+0000 7f9d3d600640 1 -- start start 2024-04-26T21:48:29.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d380b6170 0x7f9d381f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d380d1da0 0x7f9d381f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38162b80 con 0x7f9d380d19c0 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d380cfc30 con 0x7f9d380d1da0 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d3d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d38162d00 con 0x7f9d380b6170 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57002/0 (socket says 172.21.15.106:57002) 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 -- 172.21.15.106:0/1510244889 learned_addr learned my addr 172.21.15.106:0/1510244889 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 -- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d380b6170 msgr2=0x7f9d381f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d380b6170 0x7f9d381f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 -- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d380d1da0 msgr2=0x7f9d381f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d380d1da0 0x7f9d381f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.552+0000 7f9d36000640 1 -- 172.21.15.106:0/1510244889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3809f540 con 0x7f9d380d19c0 2024-04-26T21:48:29.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d36000640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9d2002a700 tx=0x7f9d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20040020 con 0x7f9d380d19c0 2024-04-26T21:48:29.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d20038040 con 0x7f9d380d19c0 2024-04-26T21:48:29.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20037070 con 0x7f9d380d19c0 2024-04-26T21:48:29.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d380a0c50 con 0x7f9d380d19c0 2024-04-26T21:48:29.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.553+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d380a1210 con 0x7f9d380d19c0 2024-04-26T21:48:29.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.555+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38069650 con 0x7f9d380d19c0 2024-04-26T21:48:29.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.556+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9d20043020 con 0x7f9d380d19c0 2024-04-26T21:48:29.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.556+0000 7f9d2ea00640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 0x7f9d1406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.557+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9d200ac570 con 0x7f9d380d19c0 2024-04-26T21:48:29.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.557+0000 7f9d36a00640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 0x7f9d1406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.557+0000 7f9d36a00640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 0x7f9d1406ef60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9d28007fd0 tx=0x7f9d28007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.561+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d2004a030 con 0x7f9d380d19c0 2024-04-26T21:48:29.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.758+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d380d26d0 con 0x7f9d380d19c0 2024-04-26T21:48:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.759+0000 7f9d2ea00640 1 -- 172.21.15.106:0/1510244889 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f9d20032e20 con 0x7f9d380d19c0 2024-04-26T21:48:29.760 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.761+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 msgr2=0x7f9d1406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.761+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 0x7f9d1406ef60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9d28007fd0 tx=0x7f9d28007900 comp rx=0 tx=0).stop 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.761+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 msgr2=0x7f9d3809edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.761+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9d2002a700 tx=0x7f9d200046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 shutdown_connections 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d380d1da0 0x7f9d381f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d380d19c0 0x7f9d3809edd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9d1406caa0 0x7f9d1406ef60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 --2- 172.21.15.106:0/1510244889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d380b6170 0x7f9d381f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 >> 172.21.15.106:0/1510244889 conn(0x7f9d3809e460 msgr2=0x7f9d3806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:29.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 shutdown_connections 2024-04-26T21:48:29.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.762+0000 7f9d3d600640 1 -- 172.21.15.106:0/1510244889 wait complete. 2024-04-26T21:48:29.780 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:29.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.976+0000 7f557a600640 1 Processor -- start 2024-04-26T21:48:29.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.976+0000 7f557a600640 1 -- start start 2024-04-26T21:48:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.977+0000 7f557a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5574156370 0x7f557415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.977+0000 7f557a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 0x7f557415de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.977+0000 7f557a600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5574156750 con 0x7f557415c760 2024-04-26T21:48:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.977+0000 7f557a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f557415d430 con 0x7f5574156370 2024-04-26T21:48:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.977+0000 7f557a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f557415cd60 con 0x7f55740d1470 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.978+0000 7f5579c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 0x7f557415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.978+0000 7f5573e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f557415c760 0x7f55740d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:52472/0 (socket says 172.21.15.106:52472) 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.978+0000 7f5573e00640 1 -- 172.21.15.106:0/1590451933 learned_addr learned my addr 172.21.15.106:0/1590451933 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.978+0000 7f5579200640 1 --2- 172.21.15.106:0/1590451933 >> v2:172.21.15.204:3300/0 conn(0x7f5574156370 0x7f557415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5573400640 1 -- 172.21.15.106:0/1590451933 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239192013 0 0) 0x7f5564001110 con 0x7f557415c760 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5573400640 1 -- 172.21.15.106:0/1590451933 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f555c003520 con 0x7f557415c760 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5579c00640 1 -- 172.21.15.106:0/1590451933 >> v2:172.21.15.204:3300/0 conn(0x7f5574156370 msgr2=0x7f557415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5579c00640 1 --2- 172.21.15.106:0/1590451933 >> v2:172.21.15.204:3300/0 conn(0x7f5574156370 0x7f557415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5579c00640 1 -- 172.21.15.106:0/1590451933 >> v1:172.21.15.204:6789/0 conn(0x7f557415c760 legacy=0x7f55740d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.979+0000 7f5579c00640 1 -- 172.21.15.106:0/1590451933 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557415d040 con 0x7f55740d1470 2024-04-26T21:48:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.980+0000 7f5579c00640 1 --2- 172.21.15.106:0/1590451933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 0x7f557415de80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=802fce6484716595 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.981+0000 7f5573400640 1 -- 172.21.15.106:0/1590451933 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c007740 con 0x7f55740d1470 2024-04-26T21:48:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.981+0000 7f5573400640 1 -- 172.21.15.106:0/1590451933 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c0078e0 con 0x7f55740d1470 2024-04-26T21:48:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.981+0000 7f5573400640 1 -- 172.21.15.106:0/1590451933 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c007c90 con 0x7f55740d1470 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.981+0000 7f557a600640 1 -- 172.21.15.106:0/1590451933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 msgr2=0x7f557415de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.981+0000 7f557a600640 1 --2- 172.21.15.106:0/1590451933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 0x7f557415de80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c02a660 comp rx=0 tx=0).stop 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 -- 172.21.15.106:0/1590451933 shutdown_connections 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 --2- 172.21.15.106:0/1590451933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55740d1470 0x7f557415de80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 --2- 172.21.15.106:0/1590451933 >> v2:172.21.15.204:3300/0 conn(0x7f5574156370 0x7f557415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 -- 172.21.15.106:0/1590451933 >> 172.21.15.106:0/1590451933 conn(0x7f557406c530 msgr2=0x7f557406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 -- 172.21.15.106:0/1590451933 shutdown_connections 2024-04-26T21:48:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.982+0000 7f557a600640 1 -- 172.21.15.106:0/1590451933 wait complete. 2024-04-26T21:48:29.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.983+0000 7f557a600640 1 Processor -- start 2024-04-26T21:48:29.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.983+0000 7f557a600640 1 -- start start 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55740d1470 0x7f55740accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 0x7f55741e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55740b3cb0 con 0x7f557415c760 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f557415d960 con 0x7f55740d1470 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.984+0000 7f557a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f557415e3c0 con 0x7f5574156370 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57010/0 (socket says 172.21.15.106:57010) 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 -- 172.21.15.106:0/3632906447 learned_addr learned my addr 172.21.15.106:0/3632906447 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579200640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55740d1470 0x7f55740accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5573e00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 0x7f55741e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 -- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 msgr2=0x7f55741e5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 0x7f55741e5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 -- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55740d1470 msgr2=0x7f55740accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55740d1470 0x7f55740accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 -- 172.21.15.106:0/3632906447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741e60a0 con 0x7f557415c760 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5573e00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 0x7f55741e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.985+0000 7f5579c00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f556c009eb0 tx=0x7f556c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.986+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c040070 con 0x7f557415c760 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.986+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c031c60 con 0x7f557415c760 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.986+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c009450 con 0x7f557415c760 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.986+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741e6380 con 0x7f557415c760 2024-04-26T21:48:29.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.986+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557414cbc0 con 0x7f557415c760 2024-04-26T21:48:29.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.987+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55740aa510 con 0x7f557415c760 2024-04-26T21:48:29.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.988+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f556c039040 con 0x7f557415c760 2024-04-26T21:48:29.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.989+0000 7f5570c00640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 0x7f555806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:29.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.989+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f556c0ad3f0 con 0x7f557415c760 2024-04-26T21:48:29.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.989+0000 7f5579200640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 0x7f555806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:29.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.990+0000 7f5579200640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 0x7f555806ef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5568009ad0 tx=0x7f5568008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:29.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:29.993+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f556c07b3f0 con 0x7f557415c760 2024-04-26T21:48:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.177+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f557406fba0 con 0x7f557415c760 2024-04-26T21:48:30.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.178+0000 7f5570c00640 1 -- 172.21.15.106:0/3632906447 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f556c07acf0 con 0x7f557415c760 2024-04-26T21:48:30.179 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.180+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 msgr2=0x7f555806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.180+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 0x7f555806ef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5568009ad0 tx=0x7f5568008040 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.180+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 msgr2=0x7f55741e5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.180+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f556c009eb0 tx=0x7f556c004a40 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 shutdown_connections 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f557415c760 0x7f55741e5a70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f555806caa0 0x7f555806ef60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5574156370 0x7f55741e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 --2- 172.21.15.106:0/3632906447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55740d1470 0x7f55740accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 >> 172.21.15.106:0/3632906447 conn(0x7f557406c530 msgr2=0x7f557406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 shutdown_connections 2024-04-26T21:48:30.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.181+0000 7f557a600640 1 -- 172.21.15.106:0/3632906447 wait complete. 2024-04-26T21:48:30.199 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:30.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.394+0000 7f3f07400640 1 Processor -- start 2024-04-26T21:48:30.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.394+0000 7f3f07400640 1 -- start start 2024-04-26T21:48:30.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f07400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3f0006ce80 0x7f3f000add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f07400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000af9a0 con 0x7f3f0015cdf0 2024-04-26T21:48:30.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f07400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000afc80 con 0x7f3f000cd650 2024-04-26T21:48:30.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f07400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000ae2c0 con 0x7f3f0006ce80 2024-04-26T21:48:30.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f04a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f0015cdf0 0x7f3f0006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43436/0 (socket says 172.21.15.106:43436) 2024-04-26T21:48:30.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.396+0000 7f3f04a00640 1 -- 172.21.15.106:0/3795659942 learned_addr learned my addr 172.21.15.106:0/3795659942 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/3795659942 >> v2:172.21.15.204:3300/0 conn(0x7f3f0006ce80 0x7f3f000add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379688061 0 0) 0x7f3f000af9a0 con 0x7f3f0015cdf0 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee4003520 con 0x7f3f0015cdf0 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1703849117 0 0) 0x7f3ee4003520 con 0x7f3f0015cdf0 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 >> v2:172.21.15.204:3300/0 conn(0x7f3f0006ce80 msgr2=0x7f3f000add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 --2- 172.21.15.106:0/3795659942 >> v2:172.21.15.204:3300/0 conn(0x7f3f0006ce80 0x7f3f000add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 >> v1:172.21.15.204:6789/0 conn(0x7f3f000cd650 legacy=0x7f3f000b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000aff60 con 0x7f3f0015cdf0 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f3ef0001670 con 0x7f3f0015cdf0 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 >> v1:172.21.15.106:6789/0 conn(0x7f3f0015cdf0 legacy=0x7f3f0006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee40039b0 0x7f3ee4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee40051c0 0x7f3ee400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400c030 con 0x7f3ee4004800 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400c1e0 con 0x7f3ee40051c0 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.397+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee400c3e0 con 0x7f3ee40039b0 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 -- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee40039b0 msgr2=0x7f3ee4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee40039b0 0x7f3ee4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 -- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee40051c0 msgr2=0x7f3ee400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee40051c0 0x7f3ee400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3ef7e00640 1 -- 172.21.15.106:0/3795659942 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee4003520 con 0x7f3ee4004800 2024-04-26T21:48:30.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.398+0000 7f3f05400640 1 -- 172.21.15.106:0/3795659942 reap_dead start 2024-04-26T21:48:30.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.399+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009ac0 tx=0x7f3ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1057f0c3b9d445c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.399+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8036070 con 0x7f3ee4004800 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.399+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8031400 con 0x7f3ee4004800 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.399+0000 7f3effe00640 1 -- 172.21.15.106:0/3795659942 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8009690 con 0x7f3ee4004800 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 -- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 msgr2=0x7f3ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009ac0 tx=0x7f3ee4005830 comp rx=0 tx=0).stop 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 -- 172.21.15.106:0/3795659942 shutdown_connections 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 --2- 172.21.15.106:0/3795659942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 -- 172.21.15.106:0/3795659942 >> 172.21.15.106:0/3795659942 conn(0x7f3f000b2d50 msgr2=0x7f3f000b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 -- 172.21.15.106:0/3795659942 shutdown_connections 2024-04-26T21:48:30.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.400+0000 7f3f07400640 1 -- 172.21.15.106:0/3795659942 wait complete. 2024-04-26T21:48:30.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.402+0000 7f3f07400640 1 Processor -- start 2024-04-26T21:48:30.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.402+0000 7f3f07400640 1 -- start start 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 0x7f3f0014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f0009d760 0x7f3f000cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0015d1d0 con 0x7f3f0009cbe0 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b05b0 con 0x7f3ee4004800 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3f07400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0015d350 con 0x7f3f0009d760 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57030/0 (socket says 172.21.15.106:57030) 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.403+0000 7f3ef7e00640 1 -- 172.21.15.106:0/2471485612 learned_addr learned my addr 172.21.15.106:0/2471485612 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:30.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3f04a00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 0x7f3f0014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3f05400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f0009d760 0x7f3f000cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 -- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f0009d760 msgr2=0x7f3f000cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f0009d760 0x7f3f000cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 -- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 msgr2=0x7f3f0014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 0x7f3f0014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 -- 172.21.15.106:0/2471485612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0009bb80 con 0x7f3f0009cbe0 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3f04a00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 0x7f3f0014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:30.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3ef7e00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ee802abb0 tx=0x7f3ee8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee803f020 con 0x7f3f0009cbe0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8037040 con 0x7f3f0009cbe0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.404+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8036070 con 0x7f3f0009cbe0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.405+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0014fcf0 con 0x7f3f0009cbe0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.405+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0009c100 con 0x7f3f0009cbe0 2024-04-26T21:48:30.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.406+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00069540 con 0x7f3f0009cbe0 2024-04-26T21:48:30.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.407+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3ee8042020 con 0x7f3f0009cbe0 2024-04-26T21:48:30.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.408+0000 7f3efd600640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 0x7f3edc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.408+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3ee80abf40 con 0x7f3f0009cbe0 2024-04-26T21:48:30.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.408+0000 7f3f04a00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 0x7f3edc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.409+0000 7f3f04a00640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 0x7f3edc06ef60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001eb0 tx=0x7f3ef0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.413+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ee8049030 con 0x7f3f0009cbe0 2024-04-26T21:48:30.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.625+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f00076760 con 0x7f3f0009cbe0 2024-04-26T21:48:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.626+0000 7f3efd600640 1 -- 172.21.15.106:0/2471485612 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f3ee807a0e0 con 0x7f3f0009cbe0 2024-04-26T21:48:30.627 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.628+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 msgr2=0x7f3edc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.628+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 0x7f3edc06ef60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0001eb0 tx=0x7f3ef0009040 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.628+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 msgr2=0x7f3f0009d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.628+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ee802abb0 tx=0x7f3ee8004730 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 shutdown_connections 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f3edc06caa0 0x7f3edc06ef60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f0009d760 0x7f3f000cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f0009cbe0 0x7f3f0009d220 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 --2- 172.21.15.106:0/2471485612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4004800 0x7f3f0014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 >> 172.21.15.106:0/2471485612 conn(0x7f3f000b2d50 msgr2=0x7f3f000d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 shutdown_connections 2024-04-26T21:48:30.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:30.629+0000 7f3f07400640 1 -- 172.21.15.106:0/2471485612 wait complete. 2024-04-26T21:48:30.647 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-04-26T21:48:30.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:30.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T21:48:30.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[41148]: fuse finished with error 0 and tester_r 0 2024-04-26T21:48:30.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:30.812 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:48:30.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:30.812 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:48:30.865 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:31.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.073+0000 7f4658c00640 1 Processor -- start 2024-04-26T21:48:31.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.073+0000 7f4658c00640 1 -- start start 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f465415cdf0 0x7f46540add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 0x7f465406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540b4f50 0x7f465406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f46540ae2c0 con 0x7f465415cdf0 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f46540b05b0 con 0x7f46540cd650 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4658c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46540af9a0 con 0x7f46540b4f50 2024-04-26T21:48:31.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4651e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f465415cdf0 0x7f46540add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4651e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f465415cdf0 0x7f46540add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57032/0 (socket says 172.21.15.106:57032) 2024-04-26T21:48:31.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4651e00640 1 -- 172.21.15.106:0/415796762 learned_addr learned my addr 172.21.15.106:0/415796762 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.075+0000 7f4651400640 1 --2- 172.21.15.106:0/415796762 >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 0x7f465406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4652800640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540b4f50 0x7f465406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651e00640 1 -- 172.21.15.106:0/415796762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540b4f50 msgr2=0x7f465406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651e00640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540b4f50 0x7f465406ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651e00640 1 -- 172.21.15.106:0/415796762 >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 msgr2=0x7f465406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651e00640 1 --2- 172.21.15.106:0/415796762 >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 0x7f465406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651e00640 1 -- 172.21.15.106:0/415796762 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46540afc80 con 0x7f465415cdf0 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.076+0000 7f4651400640 1 --2- 172.21.15.106:0/415796762 >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 0x7f465406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:31.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.077+0000 7f4651e00640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f465415cdf0 0x7f46540add80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f46540d1d00 tx=0x7f464402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260d6b9dddf35e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.077+0000 7f4650a00640 1 -- 172.21.15.106:0/415796762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644037070 con 0x7f465415cdf0 2024-04-26T21:48:31.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.077+0000 7f4650a00640 1 -- 172.21.15.106:0/415796762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4644032440 con 0x7f465415cdf0 2024-04-26T21:48:31.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.077+0000 7f4650a00640 1 -- 172.21.15.106:0/415796762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644032810 con 0x7f465415cdf0 2024-04-26T21:48:31.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 -- 172.21.15.106:0/415796762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f465415cdf0 msgr2=0x7f46540add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f465415cdf0 0x7f46540add80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f46540d1d00 tx=0x7f464402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 -- 172.21.15.106:0/415796762 shutdown_connections 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540b4f50 0x7f465406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 --2- 172.21.15.106:0/415796762 >> v2:172.21.15.204:3300/0 conn(0x7f46540cd650 0x7f465406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 --2- 172.21.15.106:0/415796762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f465415cdf0 0x7f46540add80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 -- 172.21.15.106:0/415796762 >> 172.21.15.106:0/415796762 conn(0x7f46540b2d50 msgr2=0x7f46540b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 -- 172.21.15.106:0/415796762 shutdown_connections 2024-04-26T21:48:31.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.078+0000 7f4658c00640 1 -- 172.21.15.106:0/415796762 wait complete. 2024-04-26T21:48:31.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.079+0000 7f4658c00640 1 Processor -- start 2024-04-26T21:48:31.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.080+0000 7f4658c00640 1 -- start start 2024-04-26T21:48:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540cd650 0x7f4654152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465415cdf0 0x7f4654153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f465415d1d0 con 0x7f46540b4f50 2024-04-26T21:48:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46540b0240 con 0x7f465415cdf0 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4658c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f465415d350 con 0x7f46540cd650 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57042/0 (socket says 172.21.15.106:57042) 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 -- 172.21.15.106:0/3212189648 learned_addr learned my addr 172.21.15.106:0/3212189648 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 -- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540cd650 msgr2=0x7f4654152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540cd650 0x7f4654152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 -- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465415cdf0 msgr2=0x7f4654153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465415cdf0 0x7f4654153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.081+0000 7f4651e00640 1 -- 172.21.15.106:0/3212189648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4654158030 con 0x7f46540b4f50 2024-04-26T21:48:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f4651e00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4644000c00 tx=0x7f46440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644037040 con 0x7f46540b4f50 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4644042070 con 0x7f46540b4f50 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644030bb0 con 0x7f46540b4f50 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4654158310 con 0x7f46540b4f50 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.082+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46541588d0 con 0x7f46540b4f50 2024-04-26T21:48:31.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.083+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4654069540 con 0x7f46540b4f50 2024-04-26T21:48:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.084+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f4644047460 con 0x7f46540b4f50 2024-04-26T21:48:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.085+0000 7f464a000640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 0x7f463406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.085+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f46440acbd0 con 0x7f46540b4f50 2024-04-26T21:48:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.085+0000 7f4651400640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 0x7f463406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.086+0000 7f4651400640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 0x7f463406ef60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4654073490 tx=0x7f4640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.089+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4644045360 con 0x7f46540b4f50 2024-04-26T21:48:31.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.333+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46540b49e0 con 0x7f46540b4f50 2024-04-26T21:48:31.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.334+0000 7f464a000640 1 -- 172.21.15.106:0/3212189648 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f464407ace0 con 0x7f46540b4f50 2024-04-26T21:48:31.335 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:48:31.336 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:48:31.336 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 msgr2=0x7f463406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 0x7f463406ef60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4654073490 tx=0x7f4640008040 comp rx=0 tx=0).stop 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 msgr2=0x7f465414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4644000c00 tx=0x7f46440046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 shutdown_connections 2024-04-26T21:48:31.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f465415cdf0 0x7f4654153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f463406caa0 0x7f463406ef60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46540cd650 0x7f4654152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 --2- 172.21.15.106:0/3212189648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46540b4f50 0x7f465414e0b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 >> 172.21.15.106:0/3212189648 conn(0x7f46540b2d50 msgr2=0x7f46540b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 shutdown_connections 2024-04-26T21:48:31.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.337+0000 7f4658c00640 1 -- 172.21.15.106:0/3212189648 wait complete. 2024-04-26T21:48:31.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-26T21:48:31.583 INFO:teuthology.orchestra.run.smithi106.stdout:[] 2024-04-26T21:48:31.583 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-26T21:48:31.584 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol39390 2024-04-26T21:48:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.781+0000 7fb075600640 1 Processor -- start 2024-04-26T21:48:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.782+0000 7fb075600640 1 -- start start 2024-04-26T21:48:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.783+0000 7fb075600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0701636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.783+0000 7fb075600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.783+0000 7fb075600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0700b6710 con 0x7fb0700d3110 2024-04-26T21:48:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.783+0000 7fb075600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb0700d3600 con 0x7fb0700b6210 2024-04-26T21:48:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.783+0000 7fb075600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb0700d38e0 con 0x7fb0701632e0 2024-04-26T21:48:31.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.784+0000 7fb06fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0701636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.784+0000 7fb074c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.784+0000 7fb074c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:60186/0 (socket says 172.21.15.106:60186) 2024-04-26T21:48:31.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.784+0000 7fb074c00640 1 -- 172.21.15.106:0/1758155526 learned_addr learned my addr 172.21.15.106:0/1758155526 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.785+0000 7fb06fe00640 1 -- 172.21.15.106:0/1758155526 >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 msgr2=0x7fb0700b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.785+0000 7fb06fe00640 1 --2- 172.21.15.106:0/1758155526 >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.785+0000 7fb06fe00640 1 -- 172.21.15.106:0/1758155526 >> v1:172.21.15.204:6789/0 conn(0x7fb0700d3110 legacy=0x7fb070162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.785+0000 7fb06fe00640 1 -- 172.21.15.106:0/1758155526 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0700d3bc0 con 0x7fb0701632e0 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.785+0000 7fb074c00640 1 --2- 172.21.15.106:0/1758155526 >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:48:31.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.786+0000 7fb06fe00640 1 --2- 172.21.15.106:0/1758155526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0701636c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097a0 tx=0x7fb05802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e17f07dd2fd5520 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.786+0000 7fb06ea00640 1 -- 172.21.15.106:0/1758155526 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058037070 con 0x7fb0701632e0 2024-04-26T21:48:31.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.786+0000 7fb06ea00640 1 -- 172.21.15.106:0/1758155526 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb058032440 con 0x7fb0701632e0 2024-04-26T21:48:31.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.786+0000 7fb06ea00640 1 -- 172.21.15.106:0/1758155526 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058032810 con 0x7fb0701632e0 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 -- 172.21.15.106:0/1758155526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 msgr2=0x7fb0701636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 --2- 172.21.15.106:0/1758155526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0701636c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097a0 tx=0x7fb05802a180 comp rx=0 tx=0).stop 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 -- 172.21.15.106:0/1758155526 shutdown_connections 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 --2- 172.21.15.106:0/1758155526 >> v2:172.21.15.204:3300/0 conn(0x7fb0700b6210 0x7fb0700b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 --2- 172.21.15.106:0/1758155526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0701636c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 -- 172.21.15.106:0/1758155526 >> 172.21.15.106:0/1758155526 conn(0x7fb07009e2f0 msgr2=0x7fb0700d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 -- 172.21.15.106:0/1758155526 shutdown_connections 2024-04-26T21:48:31.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.787+0000 7fb075600640 1 -- 172.21.15.106:0/1758155526 wait complete. 2024-04-26T21:48:31.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.789+0000 7fb075600640 1 Processor -- start 2024-04-26T21:48:31.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.790+0000 7fb075600640 1 -- start start 2024-04-26T21:48:31.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700b6210 0x7fb0701f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0700d3110 0x7fb0701f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0700b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb070163d80 con 0x7fb0700d3110 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb070163c00 con 0x7fb0700b6210 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb075600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb070163f00 con 0x7fb0701632e0 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb06f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0700d3110 0x7fb0701f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb06f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0700d3110 0x7fb0701f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57052/0 (socket says 172.21.15.106:57052) 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.791+0000 7fb06f400640 1 -- 172.21.15.106:0/769074645 learned_addr learned my addr 172.21.15.106:0/769074645 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06fe00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700b6210 0x7fb0701f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb074c00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0700b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 -- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 msgr2=0x7fb0700b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0701632e0 0x7fb0700b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 -- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700b6210 msgr2=0x7fb0701f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700b6210 0x7fb0701f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 -- 172.21.15.106:0/769074645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0700c9690 con 0x7fb0700d3110 2024-04-26T21:48:31.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06fe00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700b6210 0x7fb0701f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb06f400640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0700d3110 0x7fb0701f7f20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c790 tx=0x7fb06000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060016020 con 0x7fb0700d3110 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb060006d50 con 0x7fb0700d3110 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060003bb0 con 0x7fb0700d3110 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb075600640 1 -- 172.21.15.106:0/769074645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0700c9970 con 0x7fb0700d3110 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.792+0000 7fb075600640 1 -- 172.21.15.106:0/769074645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0700b4860 con 0x7fb0700d3110 2024-04-26T21:48:31.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.793+0000 7fb075600640 1 -- 172.21.15.106:0/769074645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0700693c0 con 0x7fb0700d3110 2024-04-26T21:48:31.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.794+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fb06000a850 con 0x7fb0700d3110 2024-04-26T21:48:31.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.795+0000 7fb067e00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb04c06caf0 0x7fb04c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.795+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb06008bb80 con 0x7fb0700d3110 2024-04-26T21:48:31.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.796+0000 7fb06fe00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb04c06caf0 0x7fb04c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.796+0000 7fb06fe00640 1 --2- 172.21.15.106:0/769074645 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fb04c06caf0 0x7fb04c06efb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb058034000 tx=0x7fb05802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:31.800+0000 7fb067e00640 1 -- 172.21.15.106:0/769074645 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb060020020 con 0x7fb0700d3110 2024-04-26T21:48:32.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:48:32.010+0000 7fb075600640 1 -- 172.21.15.106:0/769074645 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol39390", "target": ["mon-mgr", ""]}) -- 0x7fb0700b53b0 con 0x7fb04c06caf0 2024-04-26T21:48:45.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:45.779 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:49:15.902 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:49:15.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:49:46.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:49:46.056 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:50:16.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:50:16.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:50:31.684 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T21:50:31.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:50:31.685 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:50:31.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:50:31.895 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:50:31.896 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:50:31.896 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:50:31.896 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:31.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:50:31.954 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:50:31.954 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:50:31.954 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:32.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:50:32.011 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:50:32.011 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:50:32.011 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:32.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:50:32.066 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:50:32.066 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:50:32.066 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:32.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:50:32.123 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.329+0000 7f1842200640 1 Processor -- start 2024-04-26T21:50:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.329+0000 7f1842200640 1 -- start start 2024-04-26T21:50:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1842200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c0d0e70 0x7f183c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1842200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f183c0b3b20 0x7f183c0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1842200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c146a60 con 0x7f183c146680 2024-04-26T21:50:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1842200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f183c06e970 con 0x7f183c0b3b20 2024-04-26T21:50:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1842200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0b60a0 con 0x7f183c0d0e70 2024-04-26T21:50:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.331+0000 7f1841800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f183c0b3b20 0x7f183c0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f183c0b3b20 0x7f183c0b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46396/0 (socket says 172.21.15.106:46396) 2024-04-26T21:50:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 -- 172.21.15.106:0/3526294481 learned_addr learned my addr 172.21.15.106:0/3526294481 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1833e00640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c0d0e70 0x7f183c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 -- 172.21.15.106:0/3526294481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c0d0e70 msgr2=0x7f183c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:32.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c0d0e70 0x7f183c0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 -- 172.21.15.106:0/3526294481 >> v1:172.21.15.204:6789/0 conn(0x7f183c146680 legacy=0x7f183c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.332+0000 7f1841800640 1 -- 172.21.15.106:0/3526294481 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c06dd60 con 0x7f183c0b3b20 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.333+0000 7f1841800640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c0b3fb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1834009e50 tx=0x7f183402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e04e9f2b12359b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.333+0000 7f183be00640 1 -- 172.21.15.106:0/3526294481 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834036070 con 0x7f183c0b3b20 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.333+0000 7f183be00640 1 -- 172.21.15.106:0/3526294481 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1834031600 con 0x7f183c0b3b20 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.333+0000 7f183be00640 1 -- 172.21.15.106:0/3526294481 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18340319d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 -- 172.21.15.106:0/3526294481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 msgr2=0x7f183c0b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c0b3fb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1834009e50 tx=0x7f183402a710 comp rx=0 tx=0).stop 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 -- 172.21.15.106:0/3526294481 shutdown_connections 2024-04-26T21:50:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c0b3fb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 --2- 172.21.15.106:0/3526294481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c0d0e70 0x7f183c0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 -- 172.21.15.106:0/3526294481 >> 172.21.15.106:0/3526294481 conn(0x7f183c06c740 msgr2=0x7f183c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 -- 172.21.15.106:0/3526294481 shutdown_connections 2024-04-26T21:50:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.334+0000 7f1842200640 1 -- 172.21.15.106:0/3526294481 wait complete. 2024-04-26T21:50:32.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.336+0000 7f1842200640 1 Processor -- start 2024-04-26T21:50:32.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.336+0000 7f1842200640 1 -- start start 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 0x7f183c1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c146680 0x7f183c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f183c06e320 con 0x7f183c0b3b20 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0d17f0 con 0x7f183c0d0e70 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1842200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f183c06e600 con 0x7f183c146680 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1840e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1840e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46412/0 (socket says 172.21.15.106:46412) 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.337+0000 7f1840e00640 1 -- 172.21.15.106:0/962397354 learned_addr learned my addr 172.21.15.106:0/962397354 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1833e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 0x7f183c1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1841800640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c146680 0x7f183c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 -- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c146680 msgr2=0x7f183c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c146680 0x7f183c1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 -- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 msgr2=0x7f183c1f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 0x7f183c1f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 -- 172.21.15.106:0/962397354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c13c2f0 con 0x7f183c0b3b20 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1833e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 0x7f183c1f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.338+0000 7f1840e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c3c0 tx=0x7f182c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.339+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0052d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.339+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c005490 con 0x7f183c0b3b20 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.339+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c007260 con 0x7f183c0b3b20 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.339+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c13c5d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.339+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c13cb90 con 0x7f183c0b3b20 2024-04-26T21:50:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.340+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c0693c0 con 0x7f183c0b3b20 2024-04-26T21:50:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.341+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f182c007400 con 0x7f183c0b3b20 2024-04-26T21:50:32.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.342+0000 7f1839600640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 0x7f181806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.342+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f182c08b2d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.342+0000 7f1833e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 0x7f181806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.343+0000 7f1833e00640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 0x7f181806ef90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f183c13a0b0 tx=0x7f1824008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.347+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f182c0594d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.590+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f183c0cf0d0 con 0x7f183c0b3b20 2024-04-26T21:50:32.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.591+0000 7f1839600640 1 -- 172.21.15.106:0/962397354 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f182c058dd0 con 0x7f183c0b3b20 2024-04-26T21:50:32.592 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:32.592 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:50:32.593 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 msgr2=0x7f181806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 0x7f181806ef90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f183c13a0b0 tx=0x7f1824008040 comp rx=0 tx=0).stop 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 msgr2=0x7f183c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c3c0 tx=0x7f182c00c880 comp rx=0 tx=0).stop 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 shutdown_connections 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f181806cad0 0x7f181806ef90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f183c146680 0x7f183c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f183c0d0e70 0x7f183c1f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 --2- 172.21.15.106:0/962397354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f183c0b3b20 0x7f183c15bff0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 >> 172.21.15.106:0/962397354 conn(0x7f183c06c740 msgr2=0x7f183c15d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:32.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 shutdown_connections 2024-04-26T21:50:32.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.594+0000 7f1842200640 1 -- 172.21.15.106:0/962397354 wait complete. 2024-04-26T21:50:32.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.805+0000 7f9cef200640 1 Processor -- start 2024-04-26T21:50:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.805+0000 7f9cef200640 1 -- start start 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.806+0000 7f9cef200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.806+0000 7f9cef200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9cf00d0e70 0x7f9cf00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.806+0000 7f9cef200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cf0160930 0x7f9cf0160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.806+0000 7f9cef200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf006e970 con 0x7f9cf0160930 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cef200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf00d17f0 con 0x7f9cf00d0e70 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cef200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf0146a60 con 0x7f9cf0146680 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cf0160930 0x7f9cf0160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cf0160930 0x7f9cf0160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46416/0 (socket says 172.21.15.106:46416) 2024-04-26T21:50:32.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 -- 172.21.15.106:0/1143055264 learned_addr learned my addr 172.21.15.106:0/1143055264 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cede00640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9ced400640 1 --2- 172.21.15.106:0/1143055264 >> v2:172.21.15.204:3300/0 conn(0x7f9cf00d0e70 0x7f9cf00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 -- 172.21.15.106:0/1143055264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 msgr2=0x7f9cf00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf00b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 -- 172.21.15.106:0/1143055264 >> v2:172.21.15.204:3300/0 conn(0x7f9cf00d0e70 msgr2=0x7f9cf00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 --2- 172.21.15.106:0/1143055264 >> v2:172.21.15.204:3300/0 conn(0x7f9cf00d0e70 0x7f9cf00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.807+0000 7f9cee800640 1 -- 172.21.15.106:0/1143055264 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf006dd60 con 0x7f9cf0160930 2024-04-26T21:50:32.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.808+0000 7f9cee800640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0160dc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0074280 tx=0x7f9ce802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625291f2ac8ef4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.809+0000 7f9ceca00640 1 -- 172.21.15.106:0/1143055264 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8036070 con 0x7f9cf0160930 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.809+0000 7f9ceca00640 1 -- 172.21.15.106:0/1143055264 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8031600 con 0x7f9cf0160930 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.809+0000 7f9ceca00640 1 -- 172.21.15.106:0/1143055264 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80319d0 con 0x7f9cf0160930 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.809+0000 7f9cef200640 1 -- 172.21.15.106:0/1143055264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 msgr2=0x7f9cf0160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0160dc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0074280 tx=0x7f9ce802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 -- 172.21.15.106:0/1143055264 shutdown_connections 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0160dc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 --2- 172.21.15.106:0/1143055264 >> v2:172.21.15.204:3300/0 conn(0x7f9cf00d0e70 0x7f9cf00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 --2- 172.21.15.106:0/1143055264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 -- 172.21.15.106:0/1143055264 >> 172.21.15.106:0/1143055264 conn(0x7f9cf006c740 msgr2=0x7f9cf006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 -- 172.21.15.106:0/1143055264 shutdown_connections 2024-04-26T21:50:32.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.810+0000 7f9cef200640 1 -- 172.21.15.106:0/1143055264 wait complete. 2024-04-26T21:50:32.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.812+0000 7f9cef200640 1 Processor -- start 2024-04-26T21:50:32.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.812+0000 7f9cef200640 1 -- start start 2024-04-26T21:50:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf00d0e70 0x7f9cf015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf01362a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf006e600 con 0x7f9cf0160930 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf006e320 con 0x7f9cf00d0e70 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cef200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf00cd6a0 con 0x7f9cf0146680 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cee800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cee800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46422/0 (socket says 172.21.15.106:46422) 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cee800640 1 -- 172.21.15.106:0/3286161350 learned_addr learned my addr 172.21.15.106:0/3286161350 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9cede00640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf00d0e70 0x7f9cf015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.813+0000 7f9ced400640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf01362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 -- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 msgr2=0x7f9cf01362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf01362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 -- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf00d0e70 msgr2=0x7f9cf015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf00d0e70 0x7f9cf015c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 -- 172.21.15.106:0/3286161350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf015cb60 con 0x7f9cf0160930 2024-04-26T21:50:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cee800640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009f40 tx=0x7f9ce8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8041020 con 0x7f9cf0160930 2024-04-26T21:50:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8007780 con 0x7f9cf0160930 2024-04-26T21:50:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf01442f0 con 0x7f9cf0160930 2024-04-26T21:50:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8036040 con 0x7f9cf0160930 2024-04-26T21:50:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.814+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01448b0 con 0x7f9cf0160930 2024-04-26T21:50:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.816+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf00693c0 con 0x7f9cf0160930 2024-04-26T21:50:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.817+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9ce8044020 con 0x7f9cf0160930 2024-04-26T21:50:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.817+0000 7f9ce6000640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 0x7f9cd006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.818+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f9ce80acdb0 con 0x7f9cf0160930 2024-04-26T21:50:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.818+0000 7f9cede00640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 0x7f9cd006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.818+0000 7f9cede00640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 0x7f9cd006ef60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0006fd0 tx=0x7f9ce0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:32.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:32.822+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ce80b1070 con 0x7f9cf0160930 2024-04-26T21:50:33.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.024+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cf00733e0 con 0x7f9cf0160930 2024-04-26T21:50:33.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.027+0000 7f9ce6000640 1 -- 172.21.15.106:0/3286161350 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16776 (secure 0 0 0) 0x7f9ce804b030 con 0x7f9cf0160930 2024-04-26T21:50:33.028 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:33.029 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":47,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:39.550800+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 msgr2=0x7f9cd006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 0x7f9cd006ef60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0006fd0 tx=0x7f9ce0008040 comp rx=0 tx=0).stop 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 msgr2=0x7f9cf0145390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009f40 tx=0x7f9ce8004a40 comp rx=0 tx=0).stop 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 shutdown_connections 2024-04-26T21:50:33.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cf0160930 0x7f9cf0145390 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f9cd006caa0 0x7f9cd006ef60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cf0146680 0x7f9cf01362a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 --2- 172.21.15.106:0/3286161350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf00d0e70 0x7f9cf015c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 >> 172.21.15.106:0/3286161350 conn(0x7f9cf006c740 msgr2=0x7f9cf00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 shutdown_connections 2024-04-26T21:50:33.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.030+0000 7f9cef200640 1 -- 172.21.15.106:0/3286161350 wait complete. 2024-04-26T21:50:33.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:45:45.121445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:59.554855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:48:00.766548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:50:33.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:50:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.237+0000 7fd4cfe00640 1 Processor -- start 2024-04-26T21:50:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.237+0000 7fd4cfe00640 1 -- start start 2024-04-26T21:50:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.238+0000 7fd4cfe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d0146280 0x7fd4d01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.238+0000 7fd4cfe00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d00b4d20 con 0x7fd4d0160d60 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.238+0000 7fd4cfe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d00b5000 con 0x7fd4d0161140 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.238+0000 7fd4cfe00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d006e5f0 con 0x7fd4d0146280 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d0146280 0x7fd4d01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d0146280 0x7fd4d01466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46424/0 (socket says 172.21.15.106:46424) 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 -- 172.21.15.106:0/3038688598 learned_addr learned my addr 172.21.15.106:0/3038688598 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cea00640 1 --1- 172.21.15.106:0/3038688598 >> v1:172.21.15.106:6789/0 conn(0x7fd4d0160d60 0x7fd4d00b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46710/0 (socket says 172.21.15.106:46710) 2024-04-26T21:50:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 -- 172.21.15.106:0/3038688598 >> v1:172.21.15.204:6789/0 conn(0x7fd4d0161140 legacy=0x7fd4d006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 -- 172.21.15.106:0/3038688598 >> v1:172.21.15.106:6789/0 conn(0x7fd4d0160d60 legacy=0x7fd4d00b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.239+0000 7fd4cf400640 1 -- 172.21.15.106:0/3038688598 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00b52e0 con 0x7fd4d0146280 2024-04-26T21:50:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.240+0000 7fd4cf400640 1 --2- 172.21.15.106:0/3038688598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d01466f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e80 tx=0x7fd4c802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890ed4933513039f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.240+0000 7fd4cd600640 1 -- 172.21.15.106:0/3038688598 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8036070 con 0x7fd4d0146280 2024-04-26T21:50:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.240+0000 7fd4cd600640 1 -- 172.21.15.106:0/3038688598 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c8031600 con 0x7fd4d0146280 2024-04-26T21:50:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.240+0000 7fd4cd600640 1 -- 172.21.15.106:0/3038688598 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80319d0 con 0x7fd4d0146280 2024-04-26T21:50:33.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 -- 172.21.15.106:0/3038688598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 msgr2=0x7fd4d01466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/3038688598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d01466f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e80 tx=0x7fd4c802a7a0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 -- 172.21.15.106:0/3038688598 shutdown_connections 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/3038688598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d01466f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 -- 172.21.15.106:0/3038688598 >> 172.21.15.106:0/3038688598 conn(0x7fd4d00b2900 msgr2=0x7fd4d00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 -- 172.21.15.106:0/3038688598 shutdown_connections 2024-04-26T21:50:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.241+0000 7fd4cfe00640 1 -- 172.21.15.106:0/3038688598 wait complete. 2024-04-26T21:50:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.243+0000 7fd4cfe00640 1 Processor -- start 2024-04-26T21:50:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.243+0000 7fd4cfe00640 1 -- start start 2024-04-26T21:50:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d0160d60 0x7fd4d00ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d0161140 0x7fd4d00736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00cf6f0 con 0x7fd4d0146280 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0146c30 con 0x7fd4d0161140 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d0161770 con 0x7fd4d0160d60 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46438/0 (socket says 172.21.15.106:46438) 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.244+0000 7fd4cea00640 1 -- 172.21.15.106:0/257145086 learned_addr learned my addr 172.21.15.106:0/257145086 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 -- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d0160d60 msgr2=0x7fd4d00ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d0160d60 0x7fd4d00ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 -- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d0161140 msgr2=0x7fd4d00736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d0161140 0x7fd4d00736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 -- 172.21.15.106:0/257145086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00cb010 con 0x7fd4d0146280 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cea00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c430 tx=0x7fd4c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00074e0 con 0x7fd4d0146280 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0007680 con 0x7fd4d0146280 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d00cb2f0 con 0x7fd4d0146280 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0004a00 con 0x7fd4d0146280 2024-04-26T21:50:33.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.245+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d00cb8b0 con 0x7fd4d0146280 2024-04-26T21:50:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.246+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d00693c0 con 0x7fd4d0146280 2024-04-26T21:50:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.247+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd4c001a020 con 0x7fd4d0146280 2024-04-26T21:50:33.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.248+0000 7fd4c6a00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 0x7fd4b006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.248+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd4c008b700 con 0x7fd4d0146280 2024-04-26T21:50:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.249+0000 7fd4ce000640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 0x7fd4b006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.249+0000 7fd4ce000640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 0x7fd4b006ef60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00ab7e0 tx=0x7fd4bc007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.253+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4c0059860 con 0x7fd4d0146280 2024-04-26T21:50:33.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.458+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4d0069cd0 con 0x7fd4d0146280 2024-04-26T21:50:33.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.459+0000 7fd4c6a00640 1 -- 172.21.15.106:0/257145086 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fd4c0059160 con 0x7fd4d0146280 2024-04-26T21:50:33.460 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 msgr2=0x7fd4b006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 0x7fd4b006ef60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00ab7e0 tx=0x7fd4bc007990 comp rx=0 tx=0).stop 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 msgr2=0x7fd4d0142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c430 tx=0x7fd4c000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 shutdown_connections 2024-04-26T21:50:33.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d0161140 0x7fd4d00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd4b006caa0 0x7fd4b006ef60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d0160d60 0x7fd4d00ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 --2- 172.21.15.106:0/257145086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0146280 0x7fd4d0142d30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 >> 172.21.15.106:0/257145086 conn(0x7fd4d00b2900 msgr2=0x7fd4d00721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 shutdown_connections 2024-04-26T21:50:33.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.462+0000 7fd4cfe00640 1 -- 172.21.15.106:0/257145086 wait complete. 2024-04-26T21:50:33.481 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:50:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.675+0000 7fa532e00640 1 Processor -- start 2024-04-26T21:50:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.675+0000 7fa532e00640 1 -- start start 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.677+0000 7fa532e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.677+0000 7fa532e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa52c0d2c50 0x7fa52c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.677+0000 7fa532e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0d0170 con 0x7fa52c15fbd0 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.677+0000 7fa532e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0d0450 con 0x7fa52c0d2c50 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.677+0000 7fa532e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0d0730 con 0x7fa52c15f7f0 2024-04-26T21:50:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa52c0d2c50 0x7fa52c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa531a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa52c0d2c50 0x7fa52c0d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.106:54352/0 (socket says 172.21.15.106:54352) 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 -- 172.21.15.106:0/1072230758 learned_addr learned my addr 172.21.15.106:0/1072230758 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 -- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 msgr2=0x7fa52c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.678+0000 7fa532400640 1 -- 172.21.15.106:0/1072230758 >> v1:172.21.15.204:6789/0 conn(0x7fa52c15fbd0 legacy=0x7fa52c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.679+0000 7fa532400640 1 -- 172.21.15.106:0/1072230758 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0d0a10 con 0x7fa52c0d2c50 2024-04-26T21:50:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.679+0000 7fa531a00640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c0cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:33.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.680+0000 7fa532400640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c0d30e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa524009e50 tx=0x7fa52402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4221a983577a534c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.680+0000 7fa52be00640 1 -- 172.21.15.106:0/1072230758 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524036070 con 0x7fa52c0d2c50 2024-04-26T21:50:33.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.680+0000 7fa52be00640 1 -- 172.21.15.106:0/1072230758 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524031600 con 0x7fa52c0d2c50 2024-04-26T21:50:33.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.680+0000 7fa52be00640 1 -- 172.21.15.106:0/1072230758 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240319d0 con 0x7fa52c0d2c50 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 -- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 msgr2=0x7fa52c0d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c0d30e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa524009e50 tx=0x7fa52402a680 comp rx=0 tx=0).stop 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 -- 172.21.15.106:0/1072230758 shutdown_connections 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c0d30e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 --2- 172.21.15.106:0/1072230758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c0cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 -- 172.21.15.106:0/1072230758 >> 172.21.15.106:0/1072230758 conn(0x7fa52c0d4f80 msgr2=0x7fa52c0d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 -- 172.21.15.106:0/1072230758 shutdown_connections 2024-04-26T21:50:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.681+0000 7fa532e00640 1 -- 172.21.15.106:0/1072230758 wait complete. 2024-04-26T21:50:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.683+0000 7fa532e00640 1 Processor -- start 2024-04-26T21:50:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.683+0000 7fa532e00640 1 -- start start 2024-04-26T21:50:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c1f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c15f430 con 0x7fa52c15fbd0 2024-04-26T21:50:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c160050 con 0x7fa52c0d2c50 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c15f5b0 con 0x7fa52c15f7f0 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46442/0 (socket says 172.21.15.106:46442) 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 -- 172.21.15.106:0/1594467284 learned_addr learned my addr 172.21.15.106:0/1594467284 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa531000640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa531a00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c1f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 -- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 msgr2=0x7fa52c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 -- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 msgr2=0x7fa52c1f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c1f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa532400640 1 -- 172.21.15.106:0/1594467284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c06d030 con 0x7fa52c15fbd0 2024-04-26T21:50:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.685+0000 7fa531a00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:33.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa532400640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa524009e50 tx=0x7fa524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524041020 con 0x7fa52c15fbd0 2024-04-26T21:50:33.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240077a0 con 0x7fa52c15fbd0 2024-04-26T21:50:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524036040 con 0x7fa52c15fbd0 2024-04-26T21:50:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c06d310 con 0x7fa52c15fbd0 2024-04-26T21:50:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.686+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c06d8d0 con 0x7fa52c15fbd0 2024-04-26T21:50:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.687+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0693c0 con 0x7fa52c15fbd0 2024-04-26T21:50:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.688+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa524044020 con 0x7fa52c15fbd0 2024-04-26T21:50:33.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.689+0000 7fa529600640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 0x7fa50c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:33.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.689+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa5240acc60 con 0x7fa52c15fbd0 2024-04-26T21:50:33.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.689+0000 7fa531a00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 0x7fa50c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:33.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.690+0000 7fa531a00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 0x7fa50c06ef90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa520009ac0 tx=0x7fa520008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.694+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa52404b030 con 0x7fa52c15fbd0 2024-04-26T21:50:33.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.893+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa52c08ea90 con 0x7fa52c15fbd0 2024-04-26T21:50:33.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.894+0000 7fa529600640 1 -- 172.21.15.106:0/1594467284 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7fa52407ad70 con 0x7fa52c15fbd0 2024-04-26T21:50:33.895 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 msgr2=0x7fa50c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 0x7fa50c06ef90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa520009ac0 tx=0x7fa520008040 comp rx=0 tx=0).stop 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 msgr2=0x7fa52c15db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa524009e50 tx=0x7fa524004a40 comp rx=0 tx=0).stop 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 shutdown_connections 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa52c15fbd0 0x7fa52c15db60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa50c06cad0 0x7fa50c06ef90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa52c15f7f0 0x7fa52c06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 --2- 172.21.15.106:0/1594467284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c0d2c50 0x7fa52c1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:33.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 >> 172.21.15.106:0/1594467284 conn(0x7fa52c0d4f80 msgr2=0x7fa52c15a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:33.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 shutdown_connections 2024-04-26T21:50:33.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:33.897+0000 7fa532e00640 1 -- 172.21.15.106:0/1594467284 wait complete. 2024-04-26T21:50:33.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:50:34.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.114+0000 7f0f07e00640 1 Processor -- start 2024-04-26T21:50:34.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.114+0000 7f0f07e00640 1 -- start start 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f07e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f080b5ca0 0x7f0f080d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f07e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f08160b20 con 0x7f0f081467a0 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f07e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f08160e00 con 0x7f0f080d12a0 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f07e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0f08146b80 con 0x7f0f080b5ca0 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f080b5ca0 0x7f0f080d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f080b5ca0 0x7f0f080d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46450/0 (socket says 172.21.15.106:46450) 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 -- 172.21.15.106:0/3849244241 learned_addr learned my addr 172.21.15.106:0/3849244241 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:34.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06a00640 1 --1- 172.21.15.106:0/3849244241 >> v1:172.21.15.106:6789/0 conn(0x7f0f081467a0 0x7f0f0815dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46714/0 (socket says 172.21.15.106:46714) 2024-04-26T21:50:34.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 -- 172.21.15.106:0/3849244241 >> v1:172.21.15.204:6789/0 conn(0x7f0f080d12a0 legacy=0x7f0f080b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:34.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 -- 172.21.15.106:0/3849244241 >> v1:172.21.15.106:6789/0 conn(0x7f0f081467a0 legacy=0x7f0f0815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.116+0000 7f0f06000640 1 -- 172.21.15.106:0/3849244241 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081610e0 con 0x7f0f080b5ca0 2024-04-26T21:50:34.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.117+0000 7f0f06000640 1 --2- 172.21.15.106:0/3849244241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f080b5ca0 0x7f0f080d0d60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009870 tx=0x7f0ef402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee31f22a6964a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f05600640 1 -- 172.21.15.106:0/3849244241 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4037070 con 0x7f0f080b5ca0 2024-04-26T21:50:34.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f05600640 1 -- 172.21.15.106:0/3849244241 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4032440 con 0x7f0f080b5ca0 2024-04-26T21:50:34.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f05600640 1 -- 172.21.15.106:0/3849244241 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4032810 con 0x7f0f080b5ca0 2024-04-26T21:50:34.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 -- 172.21.15.106:0/3849244241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f080b5ca0 msgr2=0x7f0f080d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 --2- 172.21.15.106:0/3849244241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f080b5ca0 0x7f0f080d0d60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009870 tx=0x7f0ef402a2a0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 -- 172.21.15.106:0/3849244241 shutdown_connections 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 --2- 172.21.15.106:0/3849244241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f080b5ca0 0x7f0f080d0d60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 -- 172.21.15.106:0/3849244241 >> 172.21.15.106:0/3849244241 conn(0x7f0f0806c530 msgr2=0x7f0f0806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 -- 172.21.15.106:0/3849244241 shutdown_connections 2024-04-26T21:50:34.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.118+0000 7f0f07e00640 1 -- 172.21.15.106:0/3849244241 wait complete. 2024-04-26T21:50:34.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.120+0000 7f0f07e00640 1 Processor -- start 2024-04-26T21:50:34.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.120+0000 7f0f07e00640 1 -- start start 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f080b5ca0 0x7f0f0815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f080d12a0 0x7f0f08145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080d16d0 con 0x7f0f081467a0 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08161730 con 0x7f0f080d12a0 2024-04-26T21:50:34.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080cf100 con 0x7f0f080b5ca0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46458/0 (socket says 172.21.15.106:46458) 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.121+0000 7f0f07400640 1 -- 172.21.15.106:0/704367555 learned_addr learned my addr 172.21.15.106:0/704367555 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f06000640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f080d12a0 0x7f0f08145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f06a00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f080b5ca0 0x7f0f0815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 -- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f080b5ca0 msgr2=0x7f0f0815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f080b5ca0 0x7f0f0815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 -- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f080d12a0 msgr2=0x7f0f08145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f080d12a0 0x7f0f08145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 -- 172.21.15.106:0/704367555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08143930 con 0x7f0f081467a0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07400640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c780 tx=0x7f0f0000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00017070 con 0x7f0f081467a0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f00003da0 con 0x7f0f081467a0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f000052b0 con 0x7f0f081467a0 2024-04-26T21:50:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08143c10 con 0x7f0f081467a0 2024-04-26T21:50:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.122+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081441d0 con 0x7f0f081467a0 2024-04-26T21:50:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.123+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f080693c0 con 0x7f0f081467a0 2024-04-26T21:50:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.124+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0f00005450 con 0x7f0f081467a0 2024-04-26T21:50:34.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.126+0000 7f0eff400640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 0x7f0ee806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.126+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0f0008b450 con 0x7f0f081467a0 2024-04-26T21:50:34.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.126+0000 7f0f06a00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 0x7f0ee806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.127+0000 7f0f06a00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 0x7f0ee806ef60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000f60 tx=0x7f0ef8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.130+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f00059650 con 0x7f0f081467a0 2024-04-26T21:50:34.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.316+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f0806eea0 con 0x7f0f081467a0 2024-04-26T21:50:34.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.317+0000 7f0eff400640 1 -- 172.21.15.106:0/704367555 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f0f00058f50 con 0x7f0f081467a0 2024-04-26T21:50:34.318 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-26T21:50:34.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.319+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 msgr2=0x7f0ee806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.319+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 0x7f0ee806ef60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000f60 tx=0x7f0ef8009040 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.319+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 msgr2=0x7f0f081458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.319+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c780 tx=0x7f0f0000cc40 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 shutdown_connections 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f081467a0 0x7f0f081458a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f080d12a0 0x7f0f08145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ee806caa0 0x7f0ee806ef60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 --2- 172.21.15.106:0/704367555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f080b5ca0 0x7f0f0815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 >> 172.21.15.106:0/704367555 conn(0x7f0f0806c530 msgr2=0x7f0f0806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 shutdown_connections 2024-04-26T21:50:34.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.320+0000 7f0f07e00640 1 -- 172.21.15.106:0/704367555 wait complete. 2024-04-26T21:50:34.337 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.524+0000 7f1a66200640 1 Processor -- start 2024-04-26T21:50:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.524+0000 7f1a66200640 1 -- start start 2024-04-26T21:50:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a66200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a6015aba0 0x7f1a600d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a66200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 0x7f1a60072920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a66200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a60072420 con 0x7f1a6015d520 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a66200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6015af80 con 0x7f1a6015aba0 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a66200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6015dcc0 con 0x7f1a6006fde0 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a64e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a6015aba0 0x7f1a600d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a64e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a6015aba0 0x7f1a600d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46470/0 (socket says 172.21.15.106:46470) 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a64e00640 1 -- 172.21.15.106:0/212541639 learned_addr learned my addr 172.21.15.106:0/212541639 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a5fe00640 1 --1- 172.21.15.106:0/212541639 >> v1:172.21.15.106:6789/0 conn(0x7f1a6015d520 0x7f1a6006f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46718/0 (socket says 172.21.15.106:46718) 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a64e00640 1 -- 172.21.15.106:0/212541639 >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 msgr2=0x7f1a60072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a65800640 1 --2- 172.21.15.106:0/212541639 >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 0x7f1a60072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a5f400640 1 -- 172.21.15.106:0/212541639 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962531825 0 0) 0x7f1a60072420 con 0x7f1a6015d520 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.526+0000 7f1a64e00640 1 --2- 172.21.15.106:0/212541639 >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 0x7f1a60072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.527+0000 7f1a64e00640 1 -- 172.21.15.106:0/212541639 >> v1:172.21.15.106:6789/0 conn(0x7f1a6015d520 legacy=0x7f1a6006f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.527+0000 7f1a64e00640 1 -- 172.21.15.106:0/212541639 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6015dfa0 con 0x7f1a6015aba0 2024-04-26T21:50:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.527+0000 7f1a65800640 1 --2- 172.21.15.106:0/212541639 >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 0x7f1a60072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a64e00640 1 --2- 172.21.15.106:0/212541639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a600d0680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1a60075fc0 tx=0x7f1a5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c7a42f4adf5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a5f400640 1 -- 172.21.15.106:0/212541639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54037070 con 0x7f1a6015aba0 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a5f400640 1 -- 172.21.15.106:0/212541639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a54032440 con 0x7f1a6015aba0 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a5f400640 1 -- 172.21.15.106:0/212541639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54032810 con 0x7f1a6015aba0 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a66200640 1 -- 172.21.15.106:0/212541639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 msgr2=0x7f1a600d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.528+0000 7f1a66200640 1 --2- 172.21.15.106:0/212541639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a600d0680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1a60075fc0 tx=0x7f1a5402a270 comp rx=0 tx=0).stop 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 -- 172.21.15.106:0/212541639 shutdown_connections 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 --2- 172.21.15.106:0/212541639 >> v2:172.21.15.204:3300/0 conn(0x7f1a6006fde0 0x7f1a60072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 --2- 172.21.15.106:0/212541639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a600d0680 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 -- 172.21.15.106:0/212541639 >> 172.21.15.106:0/212541639 conn(0x7f1a600b2d40 msgr2=0x7f1a600b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 -- 172.21.15.106:0/212541639 shutdown_connections 2024-04-26T21:50:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.529+0000 7f1a66200640 1 -- 172.21.15.106:0/212541639 wait complete. 2024-04-26T21:50:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.530+0000 7f1a66200640 1 Processor -- start 2024-04-26T21:50:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.530+0000 7f1a66200640 1 -- start start 2024-04-26T21:50:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 0x7f1a6015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a6015d520 0x7f1a6015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6015d9b0 con 0x7f1a6015aba0 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600cf190 con 0x7f1a6006fde0 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a66200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a60072e60 con 0x7f1a6015d520 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a5fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a5fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46486/0 (socket says 172.21.15.106:46486) 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.531+0000 7f1a5fe00640 1 -- 172.21.15.106:0/3996145204 learned_addr learned my addr 172.21.15.106:0/3996145204 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a64e00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 0x7f1a6015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 -- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a6015d520 msgr2=0x7f1a6015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a6015d520 0x7f1a6015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 -- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 msgr2=0x7f1a6015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 0x7f1a6015ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 -- 172.21.15.106:0/3996145204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6015eea0 con 0x7f1a6015aba0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a64e00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 0x7f1a6015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5fe00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00c460 tx=0x7f1a4c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c018070 con 0x7f1a6015aba0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4c007500 con 0x7f1a6015aba0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c004a20 con 0x7f1a6015aba0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.532+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6015f180 con 0x7f1a6015aba0 2024-04-26T21:50:34.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.533+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6015f740 con 0x7f1a6015aba0 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.534+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a600693c0 con 0x7f1a6015aba0 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.535+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1a4c004bc0 con 0x7f1a6015aba0 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.535+0000 7f1a5cc00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 0x7f1a4006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.535+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f1a4c08b190 con 0x7f1a6015aba0 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.536+0000 7f1a64e00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 0x7f1a4006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.536+0000 7f1a64e00640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 0x7f1a4006ef60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1a54034000 tx=0x7f1a5402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.540+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a4c0592f0 con 0x7f1a6015aba0 2024-04-26T21:50:34.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.760+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a60070690 con 0x7f1a6015aba0 2024-04-26T21:50:34.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.761+0000 7f1a5cc00640 1 -- 172.21.15.106:0/3996145204 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f1a4c058bf0 con 0x7f1a6015aba0 2024-04-26T21:50:34.762 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:34.763 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:50:34.763 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-26T21:50:34.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 msgr2=0x7f1a4006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 0x7f1a4006ef60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1a54034000 tx=0x7f1a5402e000 comp rx=0 tx=0).stop 2024-04-26T21:50:34.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 msgr2=0x7f1a6015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00c460 tx=0x7f1a4c00c920 comp rx=0 tx=0).stop 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 shutdown_connections 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1a4006caa0 0x7f1a4006ef60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a6015d520 0x7f1a6015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6015aba0 0x7f1a6015bd80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 --2- 172.21.15.106:0/3996145204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a6006fde0 0x7f1a6015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 >> 172.21.15.106:0/3996145204 conn(0x7f1a600b2d40 msgr2=0x7f1a600731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.764+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 shutdown_connections 2024-04-26T21:50:34.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.765+0000 7f1a66200640 1 -- 172.21.15.106:0/3996145204 wait complete. 2024-04-26T21:50:34.783 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:34.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.976+0000 7fad88000640 1 Processor -- start 2024-04-26T21:50:34.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.976+0000 7fad88000640 1 -- start start 2024-04-26T21:50:34.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad88000640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad8015cdf0 0x7fad800add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad88000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad800cd650 0x7fad8006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad88000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800af9a0 con 0x7fad800b4f50 2024-04-26T21:50:34.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad88000640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fad800ae2c0 con 0x7fad8015cdf0 2024-04-26T21:50:34.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad88000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fad800b05b0 con 0x7fad800cd650 2024-04-26T21:50:34.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.977+0000 7fad85600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad8015cdf0 0x7fad800add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad86000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad800b4f50 0x7fad80072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46724/0 (socket says 172.21.15.106:46724) 2024-04-26T21:50:34.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad86000640 1 -- 172.21.15.106:0/2382991947 learned_addr learned my addr 172.21.15.106:0/2382991947 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:34.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad84c00640 1 --2- 172.21.15.106:0/2382991947 >> v2:172.21.15.204:3300/0 conn(0x7fad800cd650 0x7fad8006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad85600640 1 -- 172.21.15.106:0/2382991947 >> v2:172.21.15.204:3300/0 conn(0x7fad800cd650 msgr2=0x7fad8006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:34.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad85600640 1 --2- 172.21.15.106:0/2382991947 >> v2:172.21.15.204:3300/0 conn(0x7fad800cd650 0x7fad8006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad85600640 1 -- 172.21.15.106:0/2382991947 >> v1:172.21.15.106:6789/0 conn(0x7fad800b4f50 legacy=0x7fad80072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.978+0000 7fad85600640 1 -- 172.21.15.106:0/2382991947 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad800afc80 con 0x7fad8015cdf0 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.979+0000 7fad85600640 1 --2- 172.21.15.106:0/2382991947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad8015cdf0 0x7fad800add80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fad800d1d00 tx=0x7fad7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d7efb01a13b88d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.979+0000 7fad7fe00640 1 -- 172.21.15.106:0/2382991947 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74037070 con 0x7fad8015cdf0 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.979+0000 7fad7fe00640 1 -- 172.21.15.106:0/2382991947 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74032440 con 0x7fad8015cdf0 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.979+0000 7fad7fe00640 1 -- 172.21.15.106:0/2382991947 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74032810 con 0x7fad8015cdf0 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 -- 172.21.15.106:0/2382991947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad8015cdf0 msgr2=0x7fad800add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 --2- 172.21.15.106:0/2382991947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad8015cdf0 0x7fad800add80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fad800d1d00 tx=0x7fad7402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 -- 172.21.15.106:0/2382991947 shutdown_connections 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 --2- 172.21.15.106:0/2382991947 >> v2:172.21.15.204:3300/0 conn(0x7fad800cd650 0x7fad8006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 --2- 172.21.15.106:0/2382991947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad8015cdf0 0x7fad800add80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 -- 172.21.15.106:0/2382991947 >> 172.21.15.106:0/2382991947 conn(0x7fad800b2d50 msgr2=0x7fad800b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 -- 172.21.15.106:0/2382991947 shutdown_connections 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.980+0000 7fad88000640 1 -- 172.21.15.106:0/2382991947 wait complete. 2024-04-26T21:50:34.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.982+0000 7fad88000640 1 Processor -- start 2024-04-26T21:50:34.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.982+0000 7fad88000640 1 -- start start 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad800cd650 0x7fad8009c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8015cdf0 0x7fad80158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad8015d1d0 con 0x7fad800b4f50 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad800b0240 con 0x7fad8015cdf0 2024-04-26T21:50:34.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad88000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fad8015d350 con 0x7fad800cd650 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46506/0 (socket says 172.21.15.106:46506) 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 -- 172.21.15.106:0/3413309303 learned_addr learned my addr 172.21.15.106:0/3413309303 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 -- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad800cd650 msgr2=0x7fad8009c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad800cd650 0x7fad8009c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 -- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8015cdf0 msgr2=0x7fad80158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8015cdf0 0x7fad80158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.983+0000 7fad85600640 1 -- 172.21.15.106:0/3413309303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8009cc90 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad85600640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fad74009890 tx=0x7fad740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74037040 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74042070 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74030bb0 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8009cf70 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.984+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8009d530 con 0x7fad800b4f50 2024-04-26T21:50:34.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.985+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80069540 con 0x7fad800b4f50 2024-04-26T21:50:34.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.986+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fad74047460 con 0x7fad800b4f50 2024-04-26T21:50:34.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.987+0000 7fad7d600640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 0x7fad6006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.987+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fad740acb00 con 0x7fad800b4f50 2024-04-26T21:50:34.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.988+0000 7fad84c00640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 0x7fad6006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.988+0000 7fad84c00640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 0x7fad6006ef60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad801595a0 tx=0x7fad6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:34.992+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad74032e20 con 0x7fad800b4f50 2024-04-26T21:50:35.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.191+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad800b2280 con 0x7fad800b4f50 2024-04-26T21:50:35.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.195+0000 7fad7d600640 1 -- 172.21.15.106:0/3413309303 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16776 (secure 0 0 0) 0x7fad740ace20 con 0x7fad800b4f50 2024-04-26T21:50:35.196 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:35.197 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":47,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:39.550800+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:35.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 msgr2=0x7fad6006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 0x7fad6006ef60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad801595a0 tx=0x7fad6c008040 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 msgr2=0x7fad8014c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fad74009890 tx=0x7fad740046c0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 shutdown_connections 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8015cdf0 0x7fad80158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fad6006caa0 0x7fad6006ef60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad800cd650 0x7fad8009c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 --2- 172.21.15.106:0/3413309303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad800b4f50 0x7fad8014c780 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.197+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 >> 172.21.15.106:0/3413309303 conn(0x7fad800b2d50 msgr2=0x7fad800b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.198+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 shutdown_connections 2024-04-26T21:50:35.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.198+0000 7fad88000640 1 -- 172.21.15.106:0/3413309303 wait complete. 2024-04-26T21:50:35.217 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:50:35.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:35.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.397+0000 7f5f72000640 1 Processor -- start 2024-04-26T21:50:35.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.397+0000 7f5f72000640 1 -- start start 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f72000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 0x7f5f6c169e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f72000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c163920 con 0x7f5f6c0d5990 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f72000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c0d3f50 con 0x7f5f6c15f1b0 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f72000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0d36b0 con 0x7f5f6c0b92c0 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f63e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5f6c0d5990 0x7f5f6c163210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46736/0 (socket says 172.21.15.106:46736) 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f63e00640 1 -- 172.21.15.106:0/2597613706 learned_addr learned my addr 172.21.15.106:0/2597613706 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 253141104 0 0) 0x7f5f6c163920 con 0x7f5f6c0d5990 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f50003520 con 0x7f5f6c0d5990 2024-04-26T21:50:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.399+0000 7f5f71600640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 0x7f5f6c169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1850397870 0 0) 0x7f5f50003520 con 0x7f5f6c0d5990 2024-04-26T21:50:35.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 msgr2=0x7f5f6c169e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:35.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 0x7f5f6c169e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 >> v1:172.21.15.204:6789/0 conn(0x7f5f6c15f1b0 legacy=0x7f5f6c162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c0d3990 con 0x7f5f6c0d5990 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f5f54001670 con 0x7f5f6c0d5990 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 >> v1:172.21.15.106:6789/0 conn(0x7f5f6c0d5990 legacy=0x7f5f6c163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f50004800 0x7f5f50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 0x7f5f50005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0d36b0 con 0x7f5f500039e0 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5000bb80 con 0x7f5f50004800 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5000bd80 con 0x7f5f6c0b92c0 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f70c00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.400+0000 7f5f63e00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f50004800 0x7f5f50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f70c00640 1 -- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 msgr2=0x7f5f50005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f70c00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c0b92c0 0x7f5f50005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f70c00640 1 -- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f50004800 msgr2=0x7f5f50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f70c00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f50004800 0x7f5f50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f70c00640 1 -- 172.21.15.106:0/2597613706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f50003520 con 0x7f5f500039e0 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.401+0000 7f5f63e00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f50004800 0x7f5f50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f70c00640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f50003e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c001750 tx=0x7f5f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8177d8e740a5381e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c001370 con 0x7f5f500039e0 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c0052d0 con 0x7f5f500039e0 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f6be00640 1 -- 172.21.15.106:0/2597613706 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c02abe0 con 0x7f5f500039e0 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f72000640 1 -- 172.21.15.106:0/2597613706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 msgr2=0x7f5f50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f72000640 1 --2- 172.21.15.106:0/2597613706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f50003e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c001750 tx=0x7f5f50005830 comp rx=0 tx=0).stop 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.402+0000 7f5f71600640 1 -- 172.21.15.106:0/2597613706 reap_dead start 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.403+0000 7f5f72000640 1 -- 172.21.15.106:0/2597613706 shutdown_connections 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.403+0000 7f5f72000640 1 -- 172.21.15.106:0/2597613706 >> 172.21.15.106:0/2597613706 conn(0x7f5f6c09e230 msgr2=0x7f5f6c0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.403+0000 7f5f72000640 1 -- 172.21.15.106:0/2597613706 shutdown_connections 2024-04-26T21:50:35.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.403+0000 7f5f72000640 1 -- 172.21.15.106:0/2597613706 wait complete. 2024-04-26T21:50:35.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.404+0000 7f5f72000640 1 Processor -- start 2024-04-26T21:50:35.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.404+0000 7f5f72000640 1 -- start start 2024-04-26T21:50:35.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c14fa70 0x7f5f6c0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6c14f520 0x7f5f6c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0d33a0 con 0x7f5f500039e0 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b7d90 con 0x7f5f6c14f520 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f72000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c167e80 con 0x7f5f6c14fa70 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f70c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f70c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46518/0 (socket says 172.21.15.106:46518) 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.405+0000 7f5f70c00640 1 -- 172.21.15.106:0/1464347216 learned_addr learned my addr 172.21.15.106:0/1464347216 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 -- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c14fa70 msgr2=0x7f5f6c0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c14fa70 0x7f5f6c0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 -- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6c14f520 msgr2=0x7f5f6c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6c14f520 0x7f5f6c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 -- 172.21.15.106:0/1464347216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c0c9300 con 0x7f5f500039e0 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f70c00640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c001870 tx=0x7f5f5c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c040020 con 0x7f5f500039e0 2024-04-26T21:50:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c007680 con 0x7f5f500039e0 2024-04-26T21:50:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c15a320 con 0x7f5f500039e0 2024-04-26T21:50:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.406+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007a30 con 0x7f5f500039e0 2024-04-26T21:50:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.407+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c15a8e0 con 0x7f5f500039e0 2024-04-26T21:50:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.408+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c0693c0 con 0x7f5f500039e0 2024-04-26T21:50:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.409+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5f5c043020 con 0x7f5f500039e0 2024-04-26T21:50:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.410+0000 7f5f69600640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 0x7f5f4806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.410+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f5f5c0ac330 con 0x7f5f500039e0 2024-04-26T21:50:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.410+0000 7f5f63e00640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 0x7f5f4806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.410+0000 7f5f63e00640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 0x7f5f4806ef60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f540017e0 tx=0x7f5f54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.414+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f5c04a020 con 0x7f5f500039e0 2024-04-26T21:50:35.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.652+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f6c160850 con 0x7f5f500039e0 2024-04-26T21:50:35.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.654+0000 7f5f69600640 1 -- 172.21.15.106:0/1464347216 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f5f5c07a490 con 0x7f5f500039e0 2024-04-26T21:50:35.655 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:35.655 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:50:35.655 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-26T21:50:35.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 msgr2=0x7f5f4806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 0x7f5f4806ef60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f540017e0 tx=0x7f5f54009040 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 msgr2=0x7f5f6c14ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c001870 tx=0x7f5f5c001120 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 shutdown_connections 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6c14f520 0x7f5f6c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5f4806caa0 0x7f5f4806ef60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f6c14fa70 0x7f5f6c0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 --2- 172.21.15.106:0/1464347216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f500039e0 0x7f5f6c14ef20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 >> 172.21.15.106:0/1464347216 conn(0x7f5f6c09e230 msgr2=0x7f5f6c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 shutdown_connections 2024-04-26T21:50:35.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.658+0000 7f5f72000640 1 -- 172.21.15.106:0/1464347216 wait complete. 2024-04-26T21:50:35.677 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:35.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.873+0000 7f7dfea00640 1 Processor -- start 2024-04-26T21:50:35.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.873+0000 7f7dfea00640 1 -- start start 2024-04-26T21:50:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df815cdf0 0x7f7df80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7df80cd650 0x7f7df806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7df80b4f50 0x7f7df806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7df80ae2c0 con 0x7f7df80cd650 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7df80b05b0 con 0x7f7df80b4f50 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.875+0000 7f7dfea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7df80af9a0 con 0x7f7df815cdf0 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7df80cd650 0x7f7df806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7df80cd650 0x7f7df806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46522/0 (socket says 172.21.15.106:46522) 2024-04-26T21:50:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 -- 172.21.15.106:0/2692461988 learned_addr learned my addr 172.21.15.106:0/2692461988 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 -- 172.21.15.106:0/2692461988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df815cdf0 msgr2=0x7f7df80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 --2- 172.21.15.106:0/2692461988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df815cdf0 0x7f7df80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 -- 172.21.15.106:0/2692461988 >> v2:172.21.15.204:3300/0 conn(0x7f7df80b4f50 msgr2=0x7f7df806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 --2- 172.21.15.106:0/2692461988 >> v2:172.21.15.204:3300/0 conn(0x7f7df80b4f50 0x7f7df806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.876+0000 7f7df7400640 1 -- 172.21.15.106:0/2692461988 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df80afc80 con 0x7f7df80cd650 2024-04-26T21:50:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.877+0000 7f7df7400640 1 --2- 172.21.15.106:0/2692461988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df80cd650 0x7f7df806c770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7df80b0520 tx=0x7f7de402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6688b02d15aba56d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7df6a00640 1 -- 172.21.15.106:0/2692461988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4037070 con 0x7f7df80cd650 2024-04-26T21:50:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7df6a00640 1 -- 172.21.15.106:0/2692461988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4032440 con 0x7f7df80cd650 2024-04-26T21:50:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7df6a00640 1 -- 172.21.15.106:0/2692461988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4032810 con 0x7f7df80cd650 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 -- 172.21.15.106:0/2692461988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df80cd650 msgr2=0x7f7df806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 --2- 172.21.15.106:0/2692461988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df80cd650 0x7f7df806c770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7df80b0520 tx=0x7f7de402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 -- 172.21.15.106:0/2692461988 shutdown_connections 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 --2- 172.21.15.106:0/2692461988 >> v2:172.21.15.204:3300/0 conn(0x7f7df80b4f50 0x7f7df806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 --2- 172.21.15.106:0/2692461988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df80cd650 0x7f7df806c770 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 --2- 172.21.15.106:0/2692461988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df815cdf0 0x7f7df80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 -- 172.21.15.106:0/2692461988 >> 172.21.15.106:0/2692461988 conn(0x7f7df80b2d50 msgr2=0x7f7df80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 -- 172.21.15.106:0/2692461988 shutdown_connections 2024-04-26T21:50:35.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.878+0000 7f7dfea00640 1 -- 172.21.15.106:0/2692461988 wait complete. 2024-04-26T21:50:35.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.880+0000 7f7dfea00640 1 Processor -- start 2024-04-26T21:50:35.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.881+0000 7f7dfea00640 1 -- start start 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df80b4f50 0x7f7df8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df80cd650 0x7f7df814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7df815d1d0 con 0x7f7df815cdf0 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7df80b0240 con 0x7f7df80cd650 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7df815d350 con 0x7f7df80b4f50 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46526/0 (socket says 172.21.15.106:46526) 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.882+0000 7f7dfca00640 1 -- 172.21.15.106:0/1446982484 learned_addr learned my addr 172.21.15.106:0/1446982484 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7df7400640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df80cd650 0x7f7df814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7df7e00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df80b4f50 0x7f7df8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 -- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df80b4f50 msgr2=0x7f7df8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df80b4f50 0x7f7df8155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 -- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df80cd650 msgr2=0x7f7df814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:35.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df80cd650 0x7f7df814bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:35.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 -- 172.21.15.106:0/1446982484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df809cd60 con 0x7f7df815cdf0 2024-04-26T21:50:35.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfca00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7df000b9a0 tx=0x7f7df000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0019070 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df000cbc0 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0014440 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.883+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df809d040 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.884+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df814bb10 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.885+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df8069540 con 0x7f7df815cdf0 2024-04-26T21:50:35.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.886+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f7df001c070 con 0x7f7df815cdf0 2024-04-26T21:50:35.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.887+0000 7f7defe00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 0x7f7dd806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:35.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.887+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f7df008bcd0 con 0x7f7df815cdf0 2024-04-26T21:50:35.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.887+0000 7f7df7e00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 0x7f7dd806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:35.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.888+0000 7f7df7e00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 0x7f7dd806ef60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009890 tx=0x7f7de8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:35.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:35.891+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7df0059da0 con 0x7f7df815cdf0 2024-04-26T21:50:36.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.130+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df8074070 con 0x7f7df815cdf0 2024-04-26T21:50:36.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.131+0000 7f7defe00640 1 -- 172.21.15.106:0/1446982484 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f7df00596a0 con 0x7f7df815cdf0 2024-04-26T21:50:36.132 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:36.132 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:48:01.789843+0000","modified":"2024-04-26T21:48:04.848684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1457746115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1457746115},{"type":"v1","addr":"172.21.15.106:6837","nonce":1457746115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5778,"qdb_cluster":[5778]},"id":3}]} 2024-04-26T21:50:36.132 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-26T21:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 msgr2=0x7f7dd806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 0x7f7dd806ef60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009890 tx=0x7f7de8008040 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 msgr2=0x7f7df814b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7df000b9a0 tx=0x7f7df000be60 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 shutdown_connections 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df815cdf0 0x7f7df814b5d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df80cd650 0x7f7df814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7dd806caa0 0x7f7dd806ef60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 --2- 172.21.15.106:0/1446982484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7df80b4f50 0x7f7df8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 >> 172.21.15.106:0/1446982484 conn(0x7f7df80b2d50 msgr2=0x7f7df815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.133+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 shutdown_connections 2024-04-26T21:50:36.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.134+0000 7f7dfea00640 1 -- 172.21.15.106:0/1446982484 wait complete. 2024-04-26T21:50:36.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:36.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.346+0000 7f37c3400640 1 Processor -- start 2024-04-26T21:50:36.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.346+0000 7f37c3400640 1 -- start start 2024-04-26T21:50:36.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.347+0000 7f37c3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 0x7f37bc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.347+0000 7f37c3400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f37bc06ff70 0x7f37bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.347+0000 7f37c3400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37bc0ae5f0 con 0x7f37bc0ade30 2024-04-26T21:50:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.347+0000 7f37c3400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f37bc06ce80 con 0x7f37bc06ff70 2024-04-26T21:50:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.347+0000 7f37c3400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f37bc072010 con 0x7f37bc06fb50 2024-04-26T21:50:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f37bc06ff70 0x7f37bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f37bc06ff70 0x7f37bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46538/0 (socket says 172.21.15.106:46538) 2024-04-26T21:50:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 -- 172.21.15.106:0/2135069439 learned_addr learned my addr 172.21.15.106:0/2135069439 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37b3e00640 1 --2- 172.21.15.106:0/2135069439 >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 0x7f37bc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 -- 172.21.15.106:0/2135069439 >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 msgr2=0x7f37bc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 --2- 172.21.15.106:0/2135069439 >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 0x7f37bc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 -- 172.21.15.106:0/2135069439 >> v1:172.21.15.204:6789/0 conn(0x7f37bc0ade30 legacy=0x7f37bc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37c1400640 1 -- 172.21.15.106:0/2135069439 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc0722f0 con 0x7f37bc06ff70 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.348+0000 7f37b3e00640 1 --2- 172.21.15.106:0/2135069439 >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 0x7f37bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.349+0000 7f37c1400640 1 --2- 172.21.15.106:0/2135069439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc06ff70 0x7f37bc0735f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0b03e0 tx=0x7f37b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2436b2a877e7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.349+0000 7f37bbe00640 1 -- 172.21.15.106:0/2135069439 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4036070 con 0x7f37bc06ff70 2024-04-26T21:50:36.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37bbe00640 1 -- 172.21.15.106:0/2135069439 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b4031600 con 0x7f37bc06ff70 2024-04-26T21:50:36.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37bbe00640 1 -- 172.21.15.106:0/2135069439 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40319d0 con 0x7f37bc06ff70 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 -- 172.21.15.106:0/2135069439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc06ff70 msgr2=0x7f37bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 --2- 172.21.15.106:0/2135069439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc06ff70 0x7f37bc0735f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0b03e0 tx=0x7f37b402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 -- 172.21.15.106:0/2135069439 shutdown_connections 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 --2- 172.21.15.106:0/2135069439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc06ff70 0x7f37bc0735f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 --2- 172.21.15.106:0/2135069439 >> v2:172.21.15.204:3300/0 conn(0x7f37bc06fb50 0x7f37bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 -- 172.21.15.106:0/2135069439 >> 172.21.15.106:0/2135069439 conn(0x7f37bc0b2d50 msgr2=0x7f37bc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 -- 172.21.15.106:0/2135069439 shutdown_connections 2024-04-26T21:50:36.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.350+0000 7f37c3400640 1 -- 172.21.15.106:0/2135069439 wait complete. 2024-04-26T21:50:36.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.352+0000 7f37c3400640 1 Processor -- start 2024-04-26T21:50:36.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.352+0000 7f37c3400640 1 -- start start 2024-04-26T21:50:36.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 0x7f37bc1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37bc06ff70 0x7f37bc14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc072b90 con 0x7f37bc0ade30 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc0728b0 con 0x7f37bc06fb50 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37bc0ae210 con 0x7f37bc06ff70 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46550/0 (socket says 172.21.15.106:46550) 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.353+0000 7f37c1400640 1 -- 172.21.15.106:0/3835168139 learned_addr learned my addr 172.21.15.106:0/3835168139 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37b3e00640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37bc06ff70 0x7f37bc14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c0a00640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 0x7f37bc1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 -- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37bc06ff70 msgr2=0x7f37bc14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37bc06ff70 0x7f37bc14c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 -- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 msgr2=0x7f37bc1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 0x7f37bc1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 -- 172.21.15.106:0/3835168139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc14d010 con 0x7f37bc0ade30 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c0a00640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 0x7f37bc1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:36.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37c1400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f37b402ace0 tx=0x7f37b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4036050 con 0x7f37bc0ade30 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b4042070 con 0x7f37bc0ade30 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.354+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4009510 con 0x7f37bc0ade30 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.355+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc14d2f0 con 0x7f37bc0ade30 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.355+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc147800 con 0x7f37bc0ade30 2024-04-26T21:50:36.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.356+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc069540 con 0x7f37bc0ade30 2024-04-26T21:50:36.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.357+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f37b4007bd0 con 0x7f37bc0ade30 2024-04-26T21:50:36.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.358+0000 7f37b9600640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 0x7f379806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.358+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f37b40acfc0 con 0x7f37bc0ade30 2024-04-26T21:50:36.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.358+0000 7f37c0a00640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 0x7f379806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.359+0000 7f37c0a00640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 0x7f379806ef90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37ac003eb0 tx=0x7f37ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.362+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37b4045360 con 0x7f37bc0ade30 2024-04-26T21:50:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.564+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37bc1479d0 con 0x7f37bc0ade30 2024-04-26T21:50:36.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.566+0000 7f37b9600640 1 -- 172.21.15.106:0/3835168139 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16776 (secure 0 0 0) 0x7f37b407af90 con 0x7f37bc0ade30 2024-04-26T21:50:36.567 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:36.567 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":47,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:48:39.550800+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:59.554855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:48:00.766548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 msgr2=0x7f379806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 0x7f379806ef90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37ac003eb0 tx=0x7f37ac009040 comp rx=0 tx=0).stop 2024-04-26T21:50:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 msgr2=0x7f37bc0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f37b402ace0 tx=0x7f37b4004a40 comp rx=0 tx=0).stop 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 shutdown_connections 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37bc0ade30 0x7f37bc0b1cd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f379806cad0 0x7f379806ef90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37bc06ff70 0x7f37bc14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 --2- 172.21.15.106:0/3835168139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37bc06fb50 0x7f37bc1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.568+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 >> 172.21.15.106:0/3835168139 conn(0x7f37bc0b2d50 msgr2=0x7f37bc14ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.569+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 shutdown_connections 2024-04-26T21:50:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.569+0000 7f37c3400640 1 -- 172.21.15.106:0/3835168139 wait complete. 2024-04-26T21:50:36.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:50:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.783+0000 7f39cf000640 1 Processor -- start 2024-04-26T21:50:36.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.784+0000 7f39cf000640 1 -- start start 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39cf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39c81638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39cf000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f39c815f740 0x7f39c815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39cf000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c80b5fb0 con 0x7f39c81598a0 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39cf000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f39c8159c80 con 0x7f39c815f740 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39cf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39c8160110 con 0x7f39c806ce80 2024-04-26T21:50:36.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39c7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f39c81598a0 0x7f39c806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46750/0 (socket says 172.21.15.106:46750) 2024-04-26T21:50:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.785+0000 7f39c7e00640 1 -- 172.21.15.106:0/3772114842 learned_addr learned my addr 172.21.15.106:0/3772114842 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39cd000640 1 --2- 172.21.15.106:0/3772114842 >> v2:172.21.15.204:3300/0 conn(0x7f39c815f740 0x7f39c815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172133204 0 0) 0x7f39c80b5fb0 con 0x7f39c81598a0 2024-04-26T21:50:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39bf400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39c81638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39ac003520 con 0x7f39c81598a0 2024-04-26T21:50:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267558862 0 0) 0x7f39ac003520 con 0x7f39c81598a0 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 msgr2=0x7f39c81638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39c81638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 >> v2:172.21.15.204:3300/0 conn(0x7f39c815f740 msgr2=0x7f39c815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 --2- 172.21.15.106:0/3772114842 >> v2:172.21.15.204:3300/0 conn(0x7f39c815f740 0x7f39c815fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c8160370 con 0x7f39c81598a0 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f39b8001670 con 0x7f39c81598a0 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.786+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 >> v1:172.21.15.106:6789/0 conn(0x7f39c81598a0 legacy=0x7f39c806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39ac0039e0 0x7f39ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39ac004d40 0x7f39ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8159c80 con 0x7f39ac004d40 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8160110 con 0x7f39ac0039e0 2024-04-26T21:50:36.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39ac00bb40 con 0x7f39c806ce80 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39ac004d40 0x7f39ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39bf400640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7e00640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39ac0039e0 0x7f39ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 -- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 msgr2=0x7f39ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c806ce80 0x7f39ac004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 -- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39ac0039e0 msgr2=0x7f39ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39ac0039e0 0x7f39ac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39cd000640 1 -- 172.21.15.106:0/3772114842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac003520 con 0x7f39ac004d40 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.787+0000 7f39c7e00640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39ac0039e0 0x7f39ac003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.788+0000 7f39cd000640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39ac004d40 0x7f39ac0051c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f39c0007360 tx=0x7f39ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a895abbb5243448 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0009be0 con 0x7f39ac004d40 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c002e620 con 0x7f39ac004d40 2024-04-26T21:50:36.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39c7400640 1 -- 172.21.15.106:0/3772114842 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c002e9f0 con 0x7f39ac004d40 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39cf000640 1 -- 172.21.15.106:0/3772114842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39ac004d40 msgr2=0x7f39ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39cf000640 1 --2- 172.21.15.106:0/3772114842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39ac004d40 0x7f39ac0051c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f39c0007360 tx=0x7f39ac005830 comp rx=0 tx=0).stop 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39cd000640 1 -- 172.21.15.106:0/3772114842 reap_dead start 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39cf000640 1 -- 172.21.15.106:0/3772114842 shutdown_connections 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.789+0000 7f39cf000640 1 -- 172.21.15.106:0/3772114842 >> 172.21.15.106:0/3772114842 conn(0x7f39c80b2d50 msgr2=0x7f39c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.790+0000 7f39cf000640 1 -- 172.21.15.106:0/3772114842 shutdown_connections 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.790+0000 7f39cf000640 1 -- 172.21.15.106:0/3772114842 wait complete. 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.791+0000 7f39cf000640 1 Processor -- start 2024-04-26T21:50:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.791+0000 7f39cf000640 1 -- start start 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39ac004d40 0x7f39c809e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c80cd060 0x7f39c8147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8163390 con 0x7f39c809caa0 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81630b0 con 0x7f39c80cd060 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.792+0000 7f39cf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39c8160650 con 0x7f39ac004d40 2024-04-26T21:50:36.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46574/0 (socket says 172.21.15.106:46574) 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 -- 172.21.15.106:0/980093723 learned_addr learned my addr 172.21.15.106:0/980093723 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 -- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39ac004d40 msgr2=0x7f39c809e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39ac004d40 0x7f39c809e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 -- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c80cd060 msgr2=0x7f39c8147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39cd000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c80cd060 0x7f39c8147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c80cd060 0x7f39c8147de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 -- 172.21.15.106:0/980093723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c809cf80 con 0x7f39c809caa0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.793+0000 7f39bf400640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f39b000e810 tx=0x7f39b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.794+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0017070 con 0x7f39c809caa0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.794+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b000c6d0 con 0x7f39c809caa0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.794+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b000f040 con 0x7f39c809caa0 2024-04-26T21:50:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.794+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c809d260 con 0x7f39c809caa0 2024-04-26T21:50:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.794+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c809d820 con 0x7f39c809caa0 2024-04-26T21:50:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.795+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c815fbd0 con 0x7f39c809caa0 2024-04-26T21:50:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.796+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f39b001a070 con 0x7f39c809caa0 2024-04-26T21:50:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.797+0000 7f39c4c00640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 0x7f39a406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.797+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f39b008bcb0 con 0x7f39c809caa0 2024-04-26T21:50:36.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.797+0000 7f39c7e00640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 0x7f39a406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.798+0000 7f39c7e00640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 0x7f39a406ef60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39b80017e0 tx=0x7f39b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:36.801+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39c815fbd0 con 0x7f39c809caa0 2024-04-26T21:50:37.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.031+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f39c8069540 con 0x7f39c809caa0 2024-04-26T21:50:37.033 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-26T21:50:37.032+0000 7fca30c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:50:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.036+0000 7f39c4c00640 1 -- 172.21.15.106:0/980093723 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f39b0059d00 con 0x7f39c809caa0 2024-04-26T21:50:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:50:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.038+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 msgr2=0x7f39a406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.038+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 0x7f39a406ef60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39b80017e0 tx=0x7f39b8009040 comp rx=0 tx=0).stop 2024-04-26T21:50:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.038+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 msgr2=0x7f39c80ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.038+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f39b000e810 tx=0x7f39b000ecd0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 shutdown_connections 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c80cd060 0x7f39c8147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39c809caa0 0x7f39c80ccb20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f39a406caa0 0x7f39a406ef60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 --2- 172.21.15.106:0/980093723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39ac004d40 0x7f39c809e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 >> 172.21.15.106:0/980093723 conn(0x7f39c80b2d50 msgr2=0x7f39c80b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 shutdown_connections 2024-04-26T21:50:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.039+0000 7f39cf000640 1 -- 172.21.15.106:0/980093723 wait complete. 2024-04-26T21:50:37.051 INFO:tasks.ceph.mds.a.smithi106.stderr: -5266> 2024-04-26T21:48:06.690+0000 7f349f200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:50:37.059 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:50:37.105 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-26T21:50:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.239+0000 7f81ce000640 1 Processor -- start 2024-04-26T21:50:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.240+0000 7f81ce000640 1 -- start start 2024-04-26T21:50:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.241+0000 7f81ce000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81c80b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.241+0000 7f81ce000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c815f740 con 0x7f81c80b5320 2024-04-26T21:50:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.241+0000 7f81ce000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c80afbd0 con 0x7f81c80730c0 2024-04-26T21:50:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.241+0000 7f81ce000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f81c80afeb0 con 0x7f81c815f360 2024-04-26T21:50:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.241+0000 7f81c6a00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f81c80b5320 0x7f81c80729b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46760/0 (socket says 172.21.15.106:46760) 2024-04-26T21:50:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6a00640 1 -- 172.21.15.106:0/2188977403 learned_addr learned my addr 172.21.15.106:0/2188977403 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c7400640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81c80b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 587412817 0 0) 0x7f81c815f740 con 0x7f81c80b5320 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81ac003520 con 0x7f81c80b5320 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 547961233 0 0) 0x7f81ac003520 con 0x7f81c80b5320 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 msgr2=0x7f81c80b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81c80b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 >> v1:172.21.15.204:6789/0 conn(0x7f81c80730c0 legacy=0x7f81c806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c80b0190 con 0x7f81c80b5320 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f81b0001670 con 0x7f81c80b5320 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 >> v1:172.21.15.106:6789/0 conn(0x7f81c80b5320 legacy=0x7f81c80729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.242+0000 7f81c6000640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6000640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81ac003ff0 0x7f81ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6000640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81c80afeb0 con 0x7f81ac004d40 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac00bb50 con 0x7f81ac003ff0 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f81ac00bd50 con 0x7f81c815f360 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c6a00640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81ac003ff0 0x7f81ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7400640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 -- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 msgr2=0x7f81ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c815f360 0x7f81ac0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 -- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81ac003ff0 msgr2=0x7f81ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81ac003ff0 0x7f81ac004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.243+0000 7f81c7e00640 1 -- 172.21.15.106:0/2188977403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac003520 con 0x7f81ac004d40 2024-04-26T21:50:37.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.244+0000 7f81c7e00640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81ac0051c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f81c806e7c0 tx=0x7f81ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0cd5fc416499b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0037070 con 0x7f81ac004d40 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c002ad10 con 0x7f81ac004d40 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81c6000640 1 -- 172.21.15.106:0/2188977403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00040a0 con 0x7f81ac004d40 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 -- 172.21.15.106:0/2188977403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 msgr2=0x7f81ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 --2- 172.21.15.106:0/2188977403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81ac0051c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f81c806e7c0 tx=0x7f81ac005830 comp rx=0 tx=0).stop 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81c7e00640 1 -- 172.21.15.106:0/2188977403 reap_dead start 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 -- 172.21.15.106:0/2188977403 shutdown_connections 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 -- 172.21.15.106:0/2188977403 >> 172.21.15.106:0/2188977403 conn(0x7f81c806c350 msgr2=0x7f81c806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 -- 172.21.15.106:0/2188977403 shutdown_connections 2024-04-26T21:50:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.245+0000 7f81ce000640 1 -- 172.21.15.106:0/2188977403 wait complete. 2024-04-26T21:50:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.247+0000 7f81ce000640 1 Processor -- start 2024-04-26T21:50:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.247+0000 7f81ce000640 1 -- start start 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c814f380 0x7f81c814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c814d350 0x7f81c814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81c80b4900 con 0x7f81ac004d40 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81c80b0750 con 0x7f81c814f380 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81ce000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f81c80b0c70 con 0x7f81c814d350 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81c7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81c7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46598/0 (socket says 172.21.15.106:46598) 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.248+0000 7f81c7400640 1 -- 172.21.15.106:0/1298467676 learned_addr learned my addr 172.21.15.106:0/1298467676 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 -- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c814d350 msgr2=0x7f81c814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c814d350 0x7f81c814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 -- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c814f380 msgr2=0x7f81c814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c6a00640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c814f380 0x7f81c814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c814f380 0x7f81c814f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 -- 172.21.15.106:0/1298467676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c814fd40 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81c7400640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f81b800e760 tx=0x7f81b800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8017070 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b800c6d0 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b800f040 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.249+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c814dd20 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.250+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c814e2b0 con 0x7f81ac004d40 2024-04-26T21:50:37.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.251+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c8069650 con 0x7f81ac004d40 2024-04-26T21:50:37.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.252+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f81b801a070 con 0x7f81ac004d40 2024-04-26T21:50:37.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.252+0000 7f81bf400640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 0x7f81a406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:37.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.253+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f81b808bd20 con 0x7f81ac004d40 2024-04-26T21:50:37.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.253+0000 7f81c6a00640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 0x7f81a406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:37.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.254+0000 7f81c6a00640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 0x7f81a406ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000fd0 tx=0x7f81b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.257+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81b8059c90 con 0x7f81ac004d40 2024-04-26T21:50:37.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:37.497+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f81c80b30c0 con 0x7f81ac004d40 2024-04-26T21:50:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.039+0000 7f81bf400640 1 -- 172.21.15.106:0/1298467676 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f81b8059590 con 0x7f81ac004d40 2024-04-26T21:50:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 msgr2=0x7f81a406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 0x7f81a406ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000fd0 tx=0x7f81b0009040 comp rx=0 tx=0).stop 2024-04-26T21:50:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 msgr2=0x7f81c80d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f81b800e760 tx=0x7f81b800ec20 comp rx=0 tx=0).stop 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 shutdown_connections 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f81a406caa0 0x7f81a406ef60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81c814d350 0x7f81c814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c814f380 0x7f81c814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 --2- 172.21.15.106:0/1298467676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81ac004d40 0x7f81c80d0b10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 >> 172.21.15.106:0/1298467676 conn(0x7f81c806c350 msgr2=0x7f81c806f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 shutdown_connections 2024-04-26T21:50:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.043+0000 7f81ce000640 1 -- 172.21.15.106:0/1298467676 wait complete. 2024-04-26T21:50:38.066 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:50:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.304+0000 7f017d400640 1 Processor -- start 2024-04-26T21:50:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.304+0000 7f017d400640 1 -- start start 2024-04-26T21:50:38.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.306+0000 7f017d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f017815ee90 0x7f017806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.306+0000 7f017d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01780d1650 0x7f01780725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f017d400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01780b3f30 con 0x7f017806edb0 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f017d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f01780b3d00 con 0x7f01780d1650 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f017d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01780b4210 con 0x7f017815ee90 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f0177e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f017815ee90 0x7f017806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f017ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01780d1650 0x7f01780725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f0177400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f017806edb0 0x7f01780d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.106:47240/0 (socket says 172.21.15.106:47240) 2024-04-26T21:50:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.307+0000 7f0177400640 1 -- 172.21.15.106:0/3589579819 learned_addr learned my addr 172.21.15.106:0/3589579819 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.308+0000 7f017ca00640 1 -- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f017815ee90 msgr2=0x7f017806e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.308+0000 7f017ca00640 1 --2- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f017815ee90 0x7f017806e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.308+0000 7f017ca00640 1 -- 172.21.15.106:0/3589579819 >> v1:172.21.15.204:6789/0 conn(0x7f017806edb0 legacy=0x7f01780d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.308+0000 7f017ca00640 1 -- 172.21.15.106:0/3589579819 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780b44f0 con 0x7f01780d1650 2024-04-26T21:50:38.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.309+0000 7f017ca00640 1 --2- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01780d1650 0x7f01780725c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01700097a0 tx=0x7f017002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c7131ba256522d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.310+0000 7f0176a00640 1 -- 172.21.15.106:0/3589579819 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170037070 con 0x7f01780d1650 2024-04-26T21:50:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.310+0000 7f0176a00640 1 -- 172.21.15.106:0/3589579819 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017002ae50 con 0x7f01780d1650 2024-04-26T21:50:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.310+0000 7f0176a00640 1 -- 172.21.15.106:0/3589579819 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170032660 con 0x7f01780d1650 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 -- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01780d1650 msgr2=0x7f01780725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 --2- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01780d1650 0x7f01780725c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01700097a0 tx=0x7f017002a100 comp rx=0 tx=0).stop 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 -- 172.21.15.106:0/3589579819 shutdown_connections 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 --2- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01780d1650 0x7f01780725c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 --2- 172.21.15.106:0/3589579819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f017815ee90 0x7f017806e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 -- 172.21.15.106:0/3589579819 >> 172.21.15.106:0/3589579819 conn(0x7f017806c770 msgr2=0x7f017806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 -- 172.21.15.106:0/3589579819 shutdown_connections 2024-04-26T21:50:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.311+0000 7f017d400640 1 -- 172.21.15.106:0/3589579819 wait complete. 2024-04-26T21:50:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.312+0000 7f017d400640 1 Processor -- start 2024-04-26T21:50:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.312+0000 7f017d400640 1 -- start start 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 0x7f0178155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 0x7f017815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b4fd0 con 0x7f017815ee90 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b5f30 con 0x7f017806edb0 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.313+0000 7f017d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01781f6440 con 0x7f01780d1650 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48650/0 (socket says 172.21.15.106:48650) 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 -- 172.21.15.106:0/2987250260 learned_addr learned my addr 172.21.15.106:0/2987250260 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f0177400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 0x7f017815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f0177e00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 0x7f0178155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 -- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 msgr2=0x7f017815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 0x7f017815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 -- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 msgr2=0x7f0178155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 0x7f0178155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 -- 172.21.15.106:0/2987250260 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01781f6720 con 0x7f017815ee90 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f0177e00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 0x7f0178155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f0177400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 0x7f017815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.314+0000 7f017ca00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0170035040 tx=0x7f01700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.315+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170041070 con 0x7f017815ee90 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.315+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017003c460 con 0x7f017815ee90 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.315+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170037040 con 0x7f017815ee90 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.315+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01781f6a00 con 0x7f017815ee90 2024-04-26T21:50:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.315+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0178153f20 con 0x7f017815ee90 2024-04-26T21:50:38.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.316+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01780693c0 con 0x7f017815ee90 2024-04-26T21:50:38.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.317+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0170044070 con 0x7f017815ee90 2024-04-26T21:50:38.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.318+0000 7f016fe00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 0x7f015406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.318+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f01700ac590 con 0x7f017815ee90 2024-04-26T21:50:38.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.318+0000 7f0177e00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 0x7f015406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.319+0000 7f0177e00640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 0x7f015406ef90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0168007440 tx=0x7f016800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.322+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f017007a730 con 0x7f017815ee90 2024-04-26T21:50:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:38.522+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01780afc30 con 0x7f017815ee90 2024-04-26T21:50:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.052+0000 7f016fe00640 1 -- 172.21.15.106:0/2987250260 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f017007a030 con 0x7f017815ee90 2024-04-26T21:50:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:50:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 msgr2=0x7f015406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 0x7f015406ef90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0168007440 tx=0x7f016800b040 comp rx=0 tx=0).stop 2024-04-26T21:50:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 msgr2=0x7f017815c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0170035040 tx=0x7f01700046c0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 shutdown_connections 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f017815ee90 0x7f017815c210 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f015406cad0 0x7f015406ef90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01780d1650 0x7f017815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 --2- 172.21.15.106:0/2987250260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f017806edb0 0x7f0178155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.056+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 >> 172.21.15.106:0/2987250260 conn(0x7f017806c770 msgr2=0x7f01780b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.057+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 shutdown_connections 2024-04-26T21:50:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.057+0000 7f017d400640 1 -- 172.21.15.106:0/2987250260 wait complete. 2024-04-26T21:50:39.085 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:50:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.280+0000 7fd83be00640 1 Processor -- start 2024-04-26T21:50:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.281+0000 7fd83be00640 1 -- start start 2024-04-26T21:50:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.282+0000 7fd83be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd83c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.282+0000 7fd83be00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83c0cbfc0 con 0x7fd83c143850 2024-04-26T21:50:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.282+0000 7fd83be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83c06df60 con 0x7fd83c143470 2024-04-26T21:50:39.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.282+0000 7fd83be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd83c146ba0 con 0x7fd83c0733d0 2024-04-26T21:50:39.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.283+0000 7fd83a000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd83c143850 0x7fd83c0cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34660/0 (socket says 172.21.15.106:34660) 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.283+0000 7fd83a000640 1 -- 172.21.15.106:0/2090240115 learned_addr learned my addr 172.21.15.106:0/2090240115 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.283+0000 7fd83b400640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd83c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.283+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752308024 0 0) 0x7fd83c0cbfc0 con 0x7fd83c143850 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.283+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd820003520 con 0x7fd83c143850 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2740468017 0 0) 0x7fd820003520 con 0x7fd83c143850 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 msgr2=0x7fd83c06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd83c06fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 >> v1:172.21.15.204:6789/0 conn(0x7fd83c143470 legacy=0x7fd83c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c06e200 con 0x7fd83c143850 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd83b400640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd83c06fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fd824001670 con 0x7fd83c143850 2024-04-26T21:50:39.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 >> v1:172.21.15.106:6789/0 conn(0x7fd83c143850 legacy=0x7fd83c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd820003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd820004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820004d40 0x7fd8200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c146ba0 con 0x7fd8200039e0 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd82000ba10 con 0x7fd820004d40 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd82000bc10 con 0x7fd83c0733d0 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd83aa00640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd820003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd83a000640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd820004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.284+0000 7fd83b400640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820004d40 0x7fd8200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.285+0000 7fd83aa00640 1 -- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 msgr2=0x7fd820004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.285+0000 7fd83aa00640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0733d0 0x7fd820004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.285+0000 7fd83aa00640 1 -- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820004d40 msgr2=0x7fd8200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.285+0000 7fd83aa00640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820004d40 0x7fd8200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.285+0000 7fd83aa00640 1 -- 172.21.15.106:0/2090240115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd820003520 con 0x7fd8200039e0 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.286+0000 7fd83aa00640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd820003dc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd82c001910 tx=0x7fd820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62def2b82aa7fed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.286+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037070 con 0x7fd8200039e0 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.286+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c02ac60 con 0x7fd8200039e0 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.286+0000 7fd839600640 1 -- 172.21.15.106:0/2090240115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0014e0 con 0x7fd8200039e0 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 -- 172.21.15.106:0/2090240115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 msgr2=0x7fd820003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 --2- 172.21.15.106:0/2090240115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd820003dc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd82c001910 tx=0x7fd820005830 comp rx=0 tx=0).stop 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83b400640 1 -- 172.21.15.106:0/2090240115 reap_dead start 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 -- 172.21.15.106:0/2090240115 shutdown_connections 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 -- 172.21.15.106:0/2090240115 >> 172.21.15.106:0/2090240115 conn(0x7fd83c0d0b50 msgr2=0x7fd83c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 -- 172.21.15.106:0/2090240115 shutdown_connections 2024-04-26T21:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.287+0000 7fd83be00640 1 -- 172.21.15.106:0/2090240115 wait complete. 2024-04-26T21:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.288+0000 7fd83be00640 1 Processor -- start 2024-04-26T21:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.289+0000 7fd83be00640 1 -- start start 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c153500 0x7fd83c13ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c153980 0x7fd83c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c070180 con 0x7fd8200039e0 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c071ea0 con 0x7fd83c153980 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd83c06fb10 con 0x7fd83c153500 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48660/0 (socket says 172.21.15.106:48660) 2024-04-26T21:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 -- 172.21.15.106:0/628659399 learned_addr learned my addr 172.21.15.106:0/628659399 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83b400640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c153980 0x7fd83c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 -- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c153500 msgr2=0x7fd83c13ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c153500 0x7fd83c13ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 -- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c153980 msgr2=0x7fd83c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c153980 0x7fd83c0cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.290+0000 7fd83aa00640 1 -- 172.21.15.106:0/628659399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c1f9950 con 0x7fd8200039e0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd83aa00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd82c001910 tx=0x7fd82c0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037040 con 0x7fd8200039e0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c0052e0 con 0x7fd8200039e0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c007240 con 0x7fd8200039e0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c1f5a70 con 0x7fd8200039e0 2024-04-26T21:50:39.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.291+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c1f6030 con 0x7fd8200039e0 2024-04-26T21:50:39.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.292+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c06d1f0 con 0x7fd8200039e0 2024-04-26T21:50:39.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.293+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd82c03a070 con 0x7fd8200039e0 2024-04-26T21:50:39.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.295+0000 7fd832a00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 0x7fd81806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:39.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.295+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7fd82c0abc90 con 0x7fd8200039e0 2024-04-26T21:50:39.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.295+0000 7fd83a000640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 0x7fd81806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:39.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.296+0000 7fd83a000640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 0x7fd81806ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd824001120 tx=0x7fd824000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:39.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.299+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd83c06d1f0 con 0x7fd8200039e0 2024-04-26T21:50:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:39.502+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd83c0693c0 con 0x7fd8200039e0 2024-04-26T21:50:40.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.064+0000 7fd832a00640 1 -- 172.21.15.106:0/628659399 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7fd82c079ec0 con 0x7fd8200039e0 2024-04-26T21:50:40.065 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-26T21:50:40.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 msgr2=0x7fd81806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 0x7fd81806ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd824001120 tx=0x7fd824000ed0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 msgr2=0x7fd83c1f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd82c001910 tx=0x7fd82c0012b0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 shutdown_connections 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c153980 0x7fd83c0cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd81806caa0 0x7fd81806ef60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c153500 0x7fd83c13ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 --2- 172.21.15.106:0/628659399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8200039e0 0x7fd83c1f9410 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 >> 172.21.15.106:0/628659399 conn(0x7fd83c0d0b50 msgr2=0x7fd83c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 shutdown_connections 2024-04-26T21:50:40.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.068+0000 7fd83be00640 1 -- 172.21.15.106:0/628659399 wait complete. 2024-04-26T21:50:40.095 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:50:40.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.290+0000 7fed22400640 1 Processor -- start 2024-04-26T21:50:40.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.290+0000 7fed22400640 1 -- start start 2024-04-26T21:50:40.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed22400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed1c06ce80 0x7fed1c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed22400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c0af800 con 0x7fed1c15d100 2024-04-26T21:50:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed22400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c0ae340 con 0x7fed1c0b5ca0 2024-04-26T21:50:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed22400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c0afaa0 con 0x7fed1c06ce80 2024-04-26T21:50:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed1b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fed1c15d100 0x7fed1c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34664/0 (socket says 172.21.15.106:34664) 2024-04-26T21:50:40.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed1b400640 1 -- 172.21.15.106:0/1229073493 learned_addr learned my addr 172.21.15.106:0/1229073493 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870206720 0 0) 0x7fed1c0af800 con 0x7fed1c15d100 2024-04-26T21:50:40.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed13e00640 1 --2- 172.21.15.106:0/1229073493 >> v2:172.21.15.204:3300/0 conn(0x7fed1c06ce80 0x7fed1c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.292+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed00003520 con 0x7fed1c15d100 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3712403200 0 0) 0x7fed00003520 con 0x7fed1c15d100 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 >> v2:172.21.15.204:3300/0 conn(0x7fed1c06ce80 msgr2=0x7fed1c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 --2- 172.21.15.106:0/1229073493 >> v2:172.21.15.204:3300/0 conn(0x7fed1c06ce80 0x7fed1c0b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 >> v1:172.21.15.204:6789/0 conn(0x7fed1c0b5ca0 legacy=0x7fed1c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c0afd80 con 0x7fed1c15d100 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fed0c001670 con 0x7fed1c15d100 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 >> v1:172.21.15.106:6789/0 conn(0x7fed1c15d100 legacy=0x7fed1c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed00003a00 0x7fed00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 0x7fed0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c0afaa0 con 0x7fed000052e0 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed0000c200 con 0x7fed00003a00 2024-04-26T21:50:40.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed0000c400 con 0x7fed00004800 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.293+0000 7fed1be00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 0x7fed0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1b400640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed00003a00 0x7fed00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed13e00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1be00640 1 -- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed00004800 msgr2=0x7fed00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1be00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1be00640 1 -- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed00003a00 msgr2=0x7fed00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1be00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed00003a00 0x7fed00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.294+0000 7fed1be00640 1 -- 172.21.15.106:0/1229073493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed00003520 con 0x7fed000052e0 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.295+0000 7fed1be00640 1 -- 172.21.15.106:0/1229073493 reap_dead start 2024-04-26T21:50:40.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.295+0000 7fed1be00640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 0x7fed0000b9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fed14001e50 tx=0x7fed00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed7cdf7d8c7d2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.295+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14001a70 con 0x7fed000052e0 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.295+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed14003970 con 0x7fed000052e0 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.295+0000 7fed1aa00640 1 -- 172.21.15.106:0/1229073493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14003d60 con 0x7fed000052e0 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 -- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 msgr2=0x7fed0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 0x7fed0000b9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fed14001e50 tx=0x7fed00005830 comp rx=0 tx=0).stop 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 -- 172.21.15.106:0/1229073493 shutdown_connections 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 --2- 172.21.15.106:0/1229073493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed000052e0 0x7fed0000b9c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 -- 172.21.15.106:0/1229073493 >> 172.21.15.106:0/1229073493 conn(0x7fed1c0b2d50 msgr2=0x7fed1c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 -- 172.21.15.106:0/1229073493 shutdown_connections 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.296+0000 7fed22400640 1 -- 172.21.15.106:0/1229073493 wait complete. 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.298+0000 7fed22400640 1 Processor -- start 2024-04-26T21:50:40.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.298+0000 7fed22400640 1 -- start start 2024-04-26T21:50:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000052e0 0x7fed1c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 0x7fed1c143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c09ee50 con 0x7fed1c15dd80 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c0b0340 con 0x7fed1c144150 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed22400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c160b70 con 0x7fed000052e0 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed1be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed1be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48690/0 (socket says 172.21.15.106:48690) 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed1be00640 1 -- 172.21.15.106:0/361699120 learned_addr learned my addr 172.21.15.106:0/361699120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.299+0000 7fed1b400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000052e0 0x7fed1c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed13e00640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 0x7fed1c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 -- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000052e0 msgr2=0x7fed1c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000052e0 0x7fed1c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 -- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 msgr2=0x7fed1c143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 0x7fed1c143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 -- 172.21.15.106:0/361699120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c09cc20 con 0x7fed1c15dd80 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed13e00640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 0x7fed1c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed1be00640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fed1402a920 tx=0x7fed140014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14001a70 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1402ac70 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14006d70 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.300+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c09cf00 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.301+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c09d4c0 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.302+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c069650 con 0x7fed1c15dd80 2024-04-26T21:50:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.303+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fed14007030 con 0x7fed1c15dd80 2024-04-26T21:50:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.303+0000 7fed13400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 0x7fecf806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.304+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fed140ab420 con 0x7fed1c15dd80 2024-04-26T21:50:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.304+0000 7fed1b400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 0x7fecf806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.304+0000 7fed1b400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 0x7fecf806ef90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed0c001050 tx=0x7fed0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.308+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed14043060 con 0x7fed1c15dd80 2024-04-26T21:50:40.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.507+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fed1c073d00 con 0x7fed1c15dd80 2024-04-26T21:50:40.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.512+0000 7fed13400640 1 -- 172.21.15.106:0/361699120 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fed14079830 con 0x7fed1c15dd80 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.515+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 msgr2=0x7fecf806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.515+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 0x7fecf806ef90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed0c001050 tx=0x7fed0c009040 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.515+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 msgr2=0x7fed1c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.515+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fed1402a920 tx=0x7fed140014a0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 shutdown_connections 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1c15dd80 0x7fed1c15e200 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed1c144150 0x7fed1c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fecf806cad0 0x7fecf806ef90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 --2- 172.21.15.106:0/361699120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000052e0 0x7fed1c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 >> 172.21.15.106:0/361699120 conn(0x7fed1c0b2d50 msgr2=0x7fed1c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 shutdown_connections 2024-04-26T21:50:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.516+0000 7fed22400640 1 -- 172.21.15.106:0/361699120 wait complete. 2024-04-26T21:50:40.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-26T21:50:40.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.731+0000 7fec4be00640 1 Processor -- start 2024-04-26T21:50:40.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.731+0000 7fec4be00640 1 -- start start 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4be00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0d0e70 0x7fec4c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5660 0x7fec4c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c0d1790 con 0x7fec4c0b5d40 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4be00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fec4c0727f0 con 0x7fec4c0d0e70 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c0b4a10 con 0x7fec4c0b5660 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0d0e70 0x7fec4c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0d0e70 0x7fec4c0d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48704/0 (socket says 172.21.15.106:48704) 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.733+0000 7fec4aa00640 1 -- 172.21.15.106:0/760209858 learned_addr learned my addr 172.21.15.106:0/760209858 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.734+0000 7fec4a000640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5660 0x7fec4c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.734+0000 7fec4aa00640 1 -- 172.21.15.106:0/760209858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5660 msgr2=0x7fec4c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.734+0000 7fec4aa00640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5660 0x7fec4c0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.734+0000 7fec4aa00640 1 -- 172.21.15.106:0/760209858 >> v1:172.21.15.204:6789/0 conn(0x7fec4c0b5d40 legacy=0x7fec4c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.734+0000 7fec4aa00640 1 -- 172.21.15.106:0/760209858 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c0b4cf0 con 0x7fec4c0d0e70 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.735+0000 7fec4aa00640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c0d1250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009870 tx=0x7fec3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86730bdfbc80ada server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.735+0000 7fec49600640 1 -- 172.21.15.106:0/760209858 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c037070 con 0x7fec4c0d0e70 2024-04-26T21:50:40.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.735+0000 7fec49600640 1 -- 172.21.15.106:0/760209858 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c032440 con 0x7fec4c0d0e70 2024-04-26T21:50:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.735+0000 7fec49600640 1 -- 172.21.15.106:0/760209858 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c032810 con 0x7fec4c0d0e70 2024-04-26T21:50:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 -- 172.21.15.106:0/760209858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 msgr2=0x7fec4c0d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c0d1250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009870 tx=0x7fec3c02a180 comp rx=0 tx=0).stop 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 -- 172.21.15.106:0/760209858 shutdown_connections 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5660 0x7fec4c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 --2- 172.21.15.106:0/760209858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c0d1250 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 -- 172.21.15.106:0/760209858 >> 172.21.15.106:0/760209858 conn(0x7fec4c06c740 msgr2=0x7fec4c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 -- 172.21.15.106:0/760209858 shutdown_connections 2024-04-26T21:50:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.736+0000 7fec4be00640 1 -- 172.21.15.106:0/760209858 wait complete. 2024-04-26T21:50:40.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.738+0000 7fec4be00640 1 Processor -- start 2024-04-26T21:50:40.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.739+0000 7fec4be00640 1 -- start start 2024-04-26T21:50:40.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec4c0b5660 0x7fec4c141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 0x7fec4c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0b05b0 con 0x7fec4c0d0e70 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0b0430 con 0x7fec4c0b5660 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c15f1a0 con 0x7fec4c0b5d40 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48712/0 (socket says 172.21.15.106:48712) 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4b400640 1 -- 172.21.15.106:0/3007965750 learned_addr learned my addr 172.21.15.106:0/3007965750 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.740+0000 7fec4a000640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 0x7fec4c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4aa00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec4c0b5660 0x7fec4c141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 -- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 msgr2=0x7fec4c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 0x7fec4c154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 -- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec4c0b5660 msgr2=0x7fec4c141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec4c0b5660 0x7fec4c141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:40.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 -- 172.21.15.106:0/3007965750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c155330 con 0x7fec4c0d0e70 2024-04-26T21:50:40.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4a000640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 0x7fec4c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:40.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4b400640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b970 tx=0x7fec4400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44019070 con 0x7fec4c0d0e70 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4400cb40 con 0x7fec4c0d0e70 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44014440 con 0x7fec4c0d0e70 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c155610 con 0x7fec4c0d0e70 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.741+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c155bd0 con 0x7fec4c0d0e70 2024-04-26T21:50:40.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.742+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c15e8b0 con 0x7fec4c0d0e70 2024-04-26T21:50:40.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.744+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fec4401c070 con 0x7fec4c0d0e70 2024-04-26T21:50:40.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.745+0000 7fec42a00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 0x7fec2c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:40.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.745+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fec4408b940 con 0x7fec4c0d0e70 2024-04-26T21:50:40.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.745+0000 7fec4aa00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 0x7fec2c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:40.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.746+0000 7fec4aa00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 0x7fec2c06ef60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec3c034000 tx=0x7fec3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:40.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:40.751+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec4c15e8b0 con 0x7fec4c0d0e70 2024-04-26T21:50:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.027+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fec4c0693c0 con 0x7fec4c0d0e70 2024-04-26T21:50:41.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.067+0000 7fec42a00640 1 -- 172.21.15.106:0/3007965750 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v230) ==== 140+0+0 (secure 0 0 0) 0x7fec44059cc0 con 0x7fec4c0d0e70 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 msgr2=0x7fec2c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 0x7fec2c06ef60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec3c034000 tx=0x7fec3c02e000 comp rx=0 tx=0).stop 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 msgr2=0x7fec4c143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b970 tx=0x7fec4400be30 comp rx=0 tx=0).stop 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 shutdown_connections 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0d0e70 0x7fec4c143d70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fec2c06caa0 0x7fec2c06ef60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec4c0b5d40 0x7fec4c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 --2- 172.21.15.106:0/3007965750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec4c0b5660 0x7fec4c141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 >> 172.21.15.106:0/3007965750 conn(0x7fec4c06c740 msgr2=0x7fec4c15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 shutdown_connections 2024-04-26T21:50:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:41.070+0000 7fec4be00640 1 -- 172.21.15.106:0/3007965750 wait complete. 2024-04-26T21:50:41.094 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ERROR 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner:ERROR: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 7667, in test_connection_expiration 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "create", self.volname, subvolume) 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T21:50:41.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi106 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol39390' 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner: 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:50:41.096 INFO:tasks.cephfs_test_runner:Ran 2 tests in 245.231s 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner:ERROR: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 7667, in test_connection_expiration 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "create", self.volname, subvolume) 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T21:50:41.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi106 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol39390' 2024-04-26T21:50:41.098 INFO:tasks.cephfs_test_runner: 2024-04-26T21:50:41.101 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:50:41.363 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b2846c68420d4b9790e62b2bc4be54ca Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-26T21:50:41.365 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T21:50:41.380 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T21:50:41.481 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T21:50:41.482 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:50:41.482 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:41.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:50:41.543 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-26T21:50:41.543 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:41.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:50:41.596 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:50:41.597 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:41.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:50:41.656 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:50:41.656 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:50:41.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:50:41.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:41.897 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-04-26T21:50:41.954 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:50:41.955 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:50:41.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:41.956 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:50:42.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:50:42.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.115 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-04-26T21:50:42.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.230 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-26T21:50:42.289 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:50:42.289 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:50:42.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.290 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:50:42.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:50:42.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:50:42.448 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-26T21:50:42.560 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T21:50:42.569 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:50:42.569 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:50:42.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.742+0000 7fed08400640 1 Processor -- start 2024-04-26T21:50:42.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.742+0000 7fed08400640 1 -- start start 2024-04-26T21:50:42.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed08400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed0015ef60 0x7fed000b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed08400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fed000ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed08400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed000b4100 con 0x7fed0015f340 2024-04-26T21:50:42.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed08400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fed000b13d0 con 0x7fed0015ef60 2024-04-26T21:50:42.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed08400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed000b39e0 con 0x7fed000b5cf0 2024-04-26T21:50:42.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed05000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fed0015f340 0x7fed000b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34678/0 (socket says 172.21.15.106:34678) 2024-04-26T21:50:42.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed05000640 1 -- 172.21.15.106:0/1466725191 learned_addr learned my addr 172.21.15.106:0/1466725191 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fed06400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fed000ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.744+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181120284 0 0) 0x7fed000b4100 con 0x7fed0015f340 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece8003520 con 0x7fed0015f340 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fed05a00640 1 --2- 172.21.15.106:0/1466725191 >> v2:172.21.15.204:3300/0 conn(0x7fed0015ef60 0x7fed000b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 287606154 0 0) 0x7fece8003520 con 0x7fed0015f340 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 msgr2=0x7fed000ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fed000ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 >> v2:172.21.15.204:3300/0 conn(0x7fed0015ef60 msgr2=0x7fed000b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 --2- 172.21.15.106:0/1466725191 >> v2:172.21.15.204:3300/0 conn(0x7fed0015ef60 0x7fed000b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed000b4370 con 0x7fed0015f340 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fecec001670 con 0x7fed0015f340 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 >> v1:172.21.15.106:6789/0 conn(0x7fed0015f340 legacy=0x7fed000b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece80039b0 0x7fece8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fece8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fece8004d40 0x7fece80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b13d0 con 0x7fece8004d40 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b39e0 con 0x7fece80039b0 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fece800ba70 con 0x7fed000b5cf0 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fed05a00640 1 --2- 172.21.15.106:0/1466725191 >> v2:172.21.15.204:3300/0 conn(0x7fed0015ef60 0x7fed000b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.745+0000 7fed06400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fece8004d40 0x7fece80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed05000640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fece8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed05a00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece80039b0 0x7fece8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed06400640 1 -- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 msgr2=0x7fece8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed06400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed000b5cf0 0x7fece8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed06400640 1 -- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece80039b0 msgr2=0x7fece8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:42.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed06400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece80039b0 0x7fece8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed06400640 1 -- 172.21.15.106:0/1466725191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece8003520 con 0x7fece8004d40 2024-04-26T21:50:42.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.746+0000 7fed05a00640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece80039b0 0x7fece8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.747+0000 7fed06400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fece8004d40 0x7fece80051c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009fc0 tx=0x7fece8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6bb3287dd2e668 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8009c10 con 0x7fece8004d40 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf8036070 con 0x7fece8004d40 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fecffe00640 1 -- 172.21.15.106:0/1466725191 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8031880 con 0x7fece8004d40 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 -- 172.21.15.106:0/1466725191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fece8004d40 msgr2=0x7fece80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 --2- 172.21.15.106:0/1466725191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fece8004d40 0x7fece80051c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009fc0 tx=0x7fece8005830 comp rx=0 tx=0).stop 2024-04-26T21:50:42.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed06400640 1 -- 172.21.15.106:0/1466725191 reap_dead start 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 -- 172.21.15.106:0/1466725191 shutdown_connections 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 -- 172.21.15.106:0/1466725191 >> 172.21.15.106:0/1466725191 conn(0x7fed000b6580 msgr2=0x7fed000b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 -- 172.21.15.106:0/1466725191 shutdown_connections 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.748+0000 7fed08400640 1 -- 172.21.15.106:0/1466725191 wait complete. 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.750+0000 7fed08400640 1 Processor -- start 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.750+0000 7fed08400640 1 -- start start 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece8004d40 0x7fed00156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed0015e250 0x7fed00153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b1f70 con 0x7fed001540e0 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b62b0 con 0x7fece8004d40 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.751+0000 7fed08400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed000ae040 con 0x7fed0015e250 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48738/0 (socket says 172.21.15.106:48738) 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 -- 172.21.15.106:0/4163223772 learned_addr learned my addr 172.21.15.106:0/4163223772 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed05000640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed0015e250 0x7fed00153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed05a00640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece8004d40 0x7fed00156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 -- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed0015e250 msgr2=0x7fed00153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed0015e250 0x7fed00153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 -- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece8004d40 msgr2=0x7fed00156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece8004d40 0x7fed00156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.752+0000 7fed06400640 1 -- 172.21.15.106:0/4163223772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed00155de0 con 0x7fed001540e0 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fed06400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e200 tx=0x7fecf8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8009c10 con 0x7fed001540e0 2024-04-26T21:50:42.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf8040030 con 0x7fed001540e0 2024-04-26T21:50:42.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8003bd0 con 0x7fed001540e0 2024-04-26T21:50:42.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed001560c0 con 0x7fed001540e0 2024-04-26T21:50:42.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.753+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed00156680 con 0x7fed001540e0 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.754+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed00156880 con 0x7fed001540e0 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.755+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fecf8005d10 con 0x7fed001540e0 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.756+0000 7fecfd600640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 0x7fece006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.756+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fecf80acdd0 con 0x7fed001540e0 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.756+0000 7fed05a00640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 0x7fece006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.757+0000 7fed05a00640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 0x7fece006ef90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fecf4006fd0 tx=0x7fecf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:42.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.761+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed00156880 con 0x7fed001540e0 2024-04-26T21:50:42.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.957+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed00069540 con 0x7fece006cad0 2024-04-26T21:50:42.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.958+0000 7fecfd600640 1 -- 172.21.15.106:0/4163223772 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13490 (secure 0 0 0) 0x7fed00069540 con 0x7fece006cad0 2024-04-26T21:50:42.960 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:42.960 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 msgr2=0x7fece006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 0x7fece006ef90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fecf4006fd0 tx=0x7fecf4008040 comp rx=0 tx=0).stop 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 msgr2=0x7fed00142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e200 tx=0x7fecf8009750 comp rx=0 tx=0).stop 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 shutdown_connections 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed001540e0 0x7fed00142150 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fece006cad0 0x7fece006ef90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed0015e250 0x7fed00153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.961+0000 7fed08400640 1 --2- 172.21.15.106:0/4163223772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece8004d40 0x7fed00156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.962+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 >> 172.21.15.106:0/4163223772 conn(0x7fed000b6580 msgr2=0x7fed00068340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:42.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.962+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 shutdown_connections 2024-04-26T21:50:42.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:42.962+0000 7fed08400640 1 -- 172.21.15.106:0/4163223772 wait complete. 2024-04-26T21:50:42.982 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-26T21:50:41.186643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":230184,"kb_used_data":5692,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744536,"statfs":{"total":773094113280,"available":772858404864,"internally_reserved":0,"allocated":5828608,"data_stored":3079198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4389,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001671"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:50:40.073663+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:50:40.073663+0000","last_peered":"2024-04-26T21:50:40.073663+0000","last_clean":"2024-04-26T21:50:40.073663+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:50:40.073663+0000","last_undegraded":"2024-04-26T21:50:40.073663+0000","last_fullsized":"2024-04-26T21:50:40.073663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":233332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":236853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":233900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":236875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":239323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":829691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:50:42.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:50:43.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.183+0000 7fea4de00640 1 Processor -- start 2024-04-26T21:50:43.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.183+0000 7fea4de00640 1 -- start start 2024-04-26T21:50:43.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.185+0000 7fea4de00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea48164240 0x7fea48167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea4de00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea48164740 con 0x7fea4806ce80 2024-04-26T21:50:43.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea4de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea481671a0 con 0x7fea4815dc80 2024-04-26T21:50:43.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea4de00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fea480b5ff0 con 0x7fea48164240 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea47400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea48164240 0x7fea48167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea3e000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fea4806ce80 0x7fea48163b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34684/0 (socket says 172.21.15.106:34684) 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea47400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea48164240 0x7fea48167c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48754/0 (socket says 172.21.15.106:48754) 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.186+0000 7fea3e000640 1 -- 172.21.15.106:0/1122782965 learned_addr learned my addr 172.21.15.106:0/1122782965 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.187+0000 7fea47400640 1 -- 172.21.15.106:0/1122782965 >> v1:172.21.15.204:6789/0 conn(0x7fea4815dc80 legacy=0x7fea4806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.187+0000 7fea47400640 1 -- 172.21.15.106:0/1122782965 >> v1:172.21.15.106:6789/0 conn(0x7fea4806ce80 legacy=0x7fea48163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.187+0000 7fea47400640 1 -- 172.21.15.106:0/1122782965 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea48167480 con 0x7fea48164240 2024-04-26T21:50:43.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea47400640 1 --2- 172.21.15.106:0/1122782965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea48164240 0x7fea48167c80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea40009d90 tx=0x7fea4002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa7db30ebd9cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea46000640 1 -- 172.21.15.106:0/1122782965 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40031600 con 0x7fea48164240 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea46000640 1 -- 172.21.15.106:0/1122782965 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea400317a0 con 0x7fea48164240 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea46000640 1 -- 172.21.15.106:0/1122782965 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40031b70 con 0x7fea48164240 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea4de00640 1 -- 172.21.15.106:0/1122782965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea48164240 msgr2=0x7fea48167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.188+0000 7fea4de00640 1 --2- 172.21.15.106:0/1122782965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea48164240 0x7fea48167c80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea40009d90 tx=0x7fea4002a610 comp rx=0 tx=0).stop 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.189+0000 7fea4de00640 1 -- 172.21.15.106:0/1122782965 shutdown_connections 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.189+0000 7fea4de00640 1 --2- 172.21.15.106:0/1122782965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea48164240 0x7fea48167c80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.189+0000 7fea4de00640 1 -- 172.21.15.106:0/1122782965 >> 172.21.15.106:0/1122782965 conn(0x7fea480b2d50 msgr2=0x7fea480b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.189+0000 7fea4de00640 1 -- 172.21.15.106:0/1122782965 shutdown_connections 2024-04-26T21:50:43.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.189+0000 7fea4de00640 1 -- 172.21.15.106:0/1122782965 wait complete. 2024-04-26T21:50:43.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.190+0000 7fea4de00640 1 Processor -- start 2024-04-26T21:50:43.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 -- start start 2024-04-26T21:50:43.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4815dc80 0x7fea4809e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea48164240 0x7fea4814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea480add80 con 0x7fea4806ce80 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea48164a20 con 0x7fea4815dc80 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.191+0000 7fea4de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea480adf00 con 0x7fea48164240 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48770/0 (socket says 172.21.15.106:48770) 2024-04-26T21:50:43.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 -- 172.21.15.106:0/3248518000 learned_addr learned my addr 172.21.15.106:0/3248518000 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:43.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea47400640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea48164240 0x7fea4814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 -- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea48164240 msgr2=0x7fea4814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea3e000640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4815dc80 0x7fea4809e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea48164240 0x7fea4814dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 -- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4815dc80 msgr2=0x7fea4809e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4815dc80 0x7fea4809e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.192+0000 7fea46a00640 1 -- 172.21.15.106:0/3248518000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4814d700 con 0x7fea4806ce80 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea46a00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c430 tx=0x7fea3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38011790 con 0x7fea4806ce80 2024-04-26T21:50:43.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38017070 con 0x7fea4806ce80 2024-04-26T21:50:43.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea480cd7f0 con 0x7fea4806ce80 2024-04-26T21:50:43.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38011bb0 con 0x7fea4806ce80 2024-04-26T21:50:43.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.193+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea480cddb0 con 0x7fea4806ce80 2024-04-26T21:50:43.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.194+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4815d0d0 con 0x7fea4806ce80 2024-04-26T21:50:43.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.195+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fea3801c460 con 0x7fea4806ce80 2024-04-26T21:50:43.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.196+0000 7fea3fe00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 0x7fea2406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.196+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fea3808ba00 con 0x7fea4806ce80 2024-04-26T21:50:43.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.196+0000 7fea3e000640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 0x7fea2406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.197+0000 7fea3e000640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 0x7fea2406ef60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea4809db60 tx=0x7fea30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.200+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea38059ea0 con 0x7fea4806ce80 2024-04-26T21:50:43.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.395+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea48073fa0 con 0x7fea2406caa0 2024-04-26T21:50:43.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.397+0000 7fea3fe00640 1 -- 172.21.15.106:0/3248518000 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13490 (secure 0 0 0) 0x7fea48073fa0 con 0x7fea2406caa0 2024-04-26T21:50:43.398 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:43.399 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:50:43.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 msgr2=0x7fea2406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 0x7fea2406ef60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea4809db60 tx=0x7fea30009040 comp rx=0 tx=0).stop 2024-04-26T21:50:43.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 msgr2=0x7fea4814d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c430 tx=0x7fea3800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 shutdown_connections 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fea2406caa0 0x7fea2406ef60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea48164240 0x7fea4814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4815dc80 0x7fea4809e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 --2- 172.21.15.106:0/3248518000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4806ce80 0x7fea4814d1c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 >> 172.21.15.106:0/3248518000 conn(0x7fea480b2d50 msgr2=0x7fea48068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 shutdown_connections 2024-04-26T21:50:43.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.400+0000 7fea4de00640 1 -- 172.21.15.106:0/3248518000 wait complete. 2024-04-26T21:50:43.420 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-26T21:50:43.187181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":230184,"kb_used_data":5692,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744536,"statfs":{"total":773094113280,"available":772858404864,"internally_reserved":0,"allocated":5828608,"data_stored":3079198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4389,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.147406"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:50:40.073663+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:50:40.073663+0000","last_peered":"2024-04-26T21:50:40.073663+0000","last_clean":"2024-04-26T21:50:40.073663+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:50:40.073663+0000","last_undegraded":"2024-04-26T21:50:40.073663+0000","last_fullsized":"2024-04-26T21:50:40.073663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":233332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":236853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":233900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":236875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":239323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":829691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:50:43.421 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:50:43.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:50:43.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.619+0000 7f6c4ec00640 1 Processor -- start 2024-04-26T21:50:43.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.619+0000 7f6c4ec00640 1 -- start start 2024-04-26T21:50:43.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.620+0000 7f6c4ec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c48070950 0x7f6c48070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c480d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ec00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c48071270 con 0x7f6c480cf640 2024-04-26T21:50:43.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6c480cd940 con 0x7f6c48070950 2024-04-26T21:50:43.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c480d0070 con 0x7f6c480cede0 2024-04-26T21:50:43.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ce00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6c480cf640 0x7f6c480d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34688/0 (socket says 172.21.15.106:34688) 2024-04-26T21:50:43.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4ce00640 1 -- 172.21.15.106:0/821532037 learned_addr learned my addr 172.21.15.106:0/821532037 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:43.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4e200640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c480d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443423382 0 0) 0x7f6c48071270 con 0x7f6c480cf640 2024-04-26T21:50:43.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> v2:172.21.15.204:3300/0 conn(0x7f6c48070950 0x7f6c48070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.621+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c2c003520 con 0x7f6c480cf640 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3035929758 0 0) 0x7f6c2c003520 con 0x7f6c480cf640 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 msgr2=0x7f6c480d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c480d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 >> v2:172.21.15.204:3300/0 conn(0x7f6c48070950 msgr2=0x7f6c48070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 --2- 172.21.15.106:0/821532037 >> v2:172.21.15.204:3300/0 conn(0x7f6c48070950 0x7f6c48070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c480d0350 con 0x7f6c480cf640 2024-04-26T21:50:43.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f6c34001670 con 0x7f6c480cf640 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> v2:172.21.15.204:3300/0 conn(0x7f6c48070950 0x7f6c48070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 >> v1:172.21.15.106:6789/0 conn(0x7f6c480cf640 legacy=0x7f6c480d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2c004800 0x7f6c2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c480cd940 con 0x7f6c2c0039b0 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c480d0070 con 0x7f6c2c004800 2024-04-26T21:50:43.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c00ba10 con 0x7f6c480cede0 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.622+0000 7f6c4e200640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4ce00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2c004800 0x7f6c2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4d800640 1 -- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 msgr2=0x7f6c2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c480cede0 0x7f6c2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4d800640 1 -- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2c004800 msgr2=0x7f6c2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2c004800 0x7f6c2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4d800640 1 -- 172.21.15.106:0/821532037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c003520 con 0x7f6c2c0039b0 2024-04-26T21:50:43.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.623+0000 7f6c4ce00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2c004800 0x7f6c2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.624+0000 7f6c4d800640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c2c003dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c380098a0 tx=0x7f6c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900f2f5be9a223e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.624+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800f040 con 0x7f6c2c0039b0 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.624+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3802ad20 con 0x7f6c2c0039b0 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.624+0000 7f6c47e00640 1 -- 172.21.15.106:0/821532037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004250 con 0x7f6c2c0039b0 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 -- 172.21.15.106:0/821532037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 msgr2=0x7f6c2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/821532037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c2c003dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c380098a0 tx=0x7f6c2c005830 comp rx=0 tx=0).stop 2024-04-26T21:50:43.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4e200640 1 -- 172.21.15.106:0/821532037 reap_dead start 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 -- 172.21.15.106:0/821532037 shutdown_connections 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 -- 172.21.15.106:0/821532037 >> 172.21.15.106:0/821532037 conn(0x7f6c480b48d0 msgr2=0x7f6c480b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 -- 172.21.15.106:0/821532037 shutdown_connections 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.625+0000 7f6c4ec00640 1 -- 172.21.15.106:0/821532037 wait complete. 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.627+0000 7f6c4ec00640 1 Processor -- start 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.627+0000 7f6c4ec00640 1 -- start start 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c480b6320 0x7f6c480b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 0x7f6c481500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4815d2f0 con 0x7f6c2c0039b0 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c480cd5d0 con 0x7f6c480b6320 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c480b55a0 con 0x7f6c4814fc60 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48792/0 (socket says 172.21.15.106:48792) 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 -- 172.21.15.106:0/4149910041 learned_addr learned my addr 172.21.15.106:0/4149910041 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4e200640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 0x7f6c481500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4ce00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c480b6320 0x7f6c480b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 -- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 msgr2=0x7f6c481500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 0x7f6c481500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 -- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c480b6320 msgr2=0x7f6c480b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c480b6320 0x7f6c480b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4d800640 1 -- 172.21.15.106:0/4149910041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c48150b10 con 0x7f6c2c0039b0 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.628+0000 7f6c4e200640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 0x7f6c481500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:43.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c4d800640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6c380099c0 tx=0x7f6c38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38041070 con 0x7f6c2c0039b0 2024-04-26T21:50:43.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38006dd0 con 0x7f6c2c0039b0 2024-04-26T21:50:43.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003c50 con 0x7f6c2c0039b0 2024-04-26T21:50:43.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c480b6f00 con 0x7f6c2c0039b0 2024-04-26T21:50:43.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.629+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4814a9a0 con 0x7f6c2c0039b0 2024-04-26T21:50:43.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.630+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c48069ad0 con 0x7f6c2c0039b0 2024-04-26T21:50:43.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.631+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6c38044070 con 0x7f6c2c0039b0 2024-04-26T21:50:43.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.632+0000 7f6c45600640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 0x7f6c2806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.632+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f6c380ac1c0 con 0x7f6c2c0039b0 2024-04-26T21:50:43.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.633+0000 7f6c4ce00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 0x7f6c2806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.633+0000 7f6c4ce00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 0x7f6c2806ef90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c34000c00 tx=0x7f6c34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.637+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c3807a660 con 0x7f6c2c0039b0 2024-04-26T21:50:43.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.830+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c48150620 con 0x7f6c2806cad0 2024-04-26T21:50:43.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.832+0000 7f6c45600640 1 -- 172.21.15.106:0/4149910041 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13490 (secure 0 0 0) 0x7f6c48150620 con 0x7f6c2806cad0 2024-04-26T21:50:43.833 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:43.833 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:50:43.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 msgr2=0x7f6c2806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 0x7f6c2806ef90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c34000c00 tx=0x7f6c34009040 comp rx=0 tx=0).stop 2024-04-26T21:50:43.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 msgr2=0x7f6c4815fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6c380099c0 tx=0x7f6c38004730 comp rx=0 tx=0).stop 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 shutdown_connections 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6c2806cad0 0x7f6c2806ef90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c4814fc60 0x7f6c481500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c480b6320 0x7f6c480b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 --2- 172.21.15.106:0/4149910041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0039b0 0x7f6c4815fe80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 >> 172.21.15.106:0/4149910041 conn(0x7f6c480b48d0 msgr2=0x7f6c480c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.834+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 shutdown_connections 2024-04-26T21:50:43.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:43.835+0000 7f6c4ec00640 1 -- 172.21.15.106:0/4149910041 wait complete. 2024-04-26T21:50:43.854 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-26T21:50:43.187181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":230184,"kb_used_data":5692,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744536,"statfs":{"total":773094113280,"available":772858404864,"internally_reserved":0,"allocated":5828608,"data_stored":3079198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4389,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.147406"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:50:40.073663+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:50:40.073663+0000","last_peered":"2024-04-26T21:50:40.073663+0000","last_clean":"2024-04-26T21:50:40.073663+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:50:40.073663+0000","last_undegraded":"2024-04-26T21:50:40.073663+0000","last_fullsized":"2024-04-26T21:50:40.073663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":233332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":236853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":233900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":236875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":239323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":829691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:50:43.855 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:44.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.060+0000 7fa7e7200640 1 Processor -- start 2024-04-26T21:50:44.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.060+0000 7fa7e7200640 1 -- start start 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.061+0000 7fa7e7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e80b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.061+0000 7fa7e7200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7e80d0e70 0x7fa7e80d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7e7200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e8146a60 con 0x7fa7e8160930 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7e7200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e806e970 con 0x7fa7e80d0e70 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7e7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80d17f0 con 0x7fa7e8146680 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7dfe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7e80d0e70 0x7fa7e80d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:44.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7dfe00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa7e80d0e70 0x7fa7e80d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48802/0 (socket says 172.21.15.106:48802) 2024-04-26T21:50:44.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7dfe00640 1 -- 172.21.15.106:0/2155858723 learned_addr learned my addr 172.21.15.106:0/2155858723 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:44.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.062+0000 7fa7e5e00640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e80b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:44.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.063+0000 7fa7dfe00640 1 -- 172.21.15.106:0/2155858723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 msgr2=0x7fa7e80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:44.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.063+0000 7fa7dfe00640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e80b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.063+0000 7fa7dfe00640 1 -- 172.21.15.106:0/2155858723 >> v1:172.21.15.204:6789/0 conn(0x7fa7e8160930 legacy=0x7fa7e8160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.063+0000 7fa7dfe00640 1 -- 172.21.15.106:0/2155858723 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e806dd60 con 0x7fa7e80d0e70 2024-04-26T21:50:44.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.064+0000 7fa7dfe00640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e80d12b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009870 tx=0x7fa7d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeff987201145a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:44.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e5400640 1 -- 172.21.15.106:0/2155858723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0037070 con 0x7fa7e80d0e70 2024-04-26T21:50:44.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e5400640 1 -- 172.21.15.106:0/2155858723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0032440 con 0x7fa7e80d0e70 2024-04-26T21:50:44.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e5400640 1 -- 172.21.15.106:0/2155858723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0032810 con 0x7fa7e80d0e70 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 -- 172.21.15.106:0/2155858723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 msgr2=0x7fa7e80d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e80d12b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009870 tx=0x7fa7d002a270 comp rx=0 tx=0).stop 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 -- 172.21.15.106:0/2155858723 shutdown_connections 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e80d12b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 --2- 172.21.15.106:0/2155858723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e80b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 -- 172.21.15.106:0/2155858723 >> 172.21.15.106:0/2155858723 conn(0x7fa7e806c740 msgr2=0x7fa7e806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.065+0000 7fa7e7200640 1 -- 172.21.15.106:0/2155858723 shutdown_connections 2024-04-26T21:50:44.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.066+0000 7fa7e7200640 1 -- 172.21.15.106:0/2155858723 wait complete. 2024-04-26T21:50:44.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.067+0000 7fa7e7200640 1 Processor -- start 2024-04-26T21:50:44.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.067+0000 7fa7e7200640 1 -- start start 2024-04-26T21:50:44.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e8073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e8160930 0x7fa7e80c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e806e600 con 0x7fa7e80d0e70 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e806e320 con 0x7fa7e8160930 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.068+0000 7fa7e7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80cd6a0 con 0x7fa7e8146680 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:44.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48812/0 (socket says 172.21.15.106:48812) 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 -- 172.21.15.106:0/3576221828 learned_addr learned my addr 172.21.15.106:0/3576221828 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 -- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 msgr2=0x7fa7e8073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e8073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 -- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e8160930 msgr2=0x7fa7e80c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e8160930 0x7fa7e80c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 -- 172.21.15.106:0/3576221828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e8072290 con 0x7fa7e80d0e70 2024-04-26T21:50:44.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7e5e00640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e9b0 tx=0x7fa7d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:44.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.069+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8017070 con 0x7fa7e80d0e70 2024-04-26T21:50:44.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.070+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d800c9a0 con 0x7fa7e80d0e70 2024-04-26T21:50:44.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.070+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800cd90 con 0x7fa7e80d0e70 2024-04-26T21:50:44.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.070+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e8072570 con 0x7fa7e80d0e70 2024-04-26T21:50:44.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.070+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e8072b30 con 0x7fa7e80d0e70 2024-04-26T21:50:44.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.071+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e80693c0 con 0x7fa7e80d0e70 2024-04-26T21:50:44.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.072+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa7d801a070 con 0x7fa7e80d0e70 2024-04-26T21:50:44.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.073+0000 7fa7de000640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 0x7fa7c406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:44.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.073+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa7d808bfa0 con 0x7fa7e80d0e70 2024-04-26T21:50:44.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.073+0000 7fa7dfe00640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 0x7fa7c406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:44.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.074+0000 7fa7dfe00640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 0x7fa7c406ef60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0034000 tx=0x7fa7d002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:44.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.077+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7d805a3b0 con 0x7fa7e80d0e70 2024-04-26T21:50:44.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.280+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7e815f240 con 0x7fa7e80d0e70 2024-04-26T21:50:44.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.282+0000 7fa7de000640 1 -- 172.21.15.106:0/3576221828 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13034 (secure 0 0 0) 0x7fa7d8059cb0 con 0x7fa7e80d0e70 2024-04-26T21:50:44.283 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:44.284 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":50,"fsid":"e6ee84ed-cbe5-4ff1-aa0d-97a291446523","created":"2024-04-26T21:45:32.013738+0000","modified":"2024-04-26T21:50:40.054347+0000","last_up_change":"2024-04-26T21:45:44.446193+0000","last_in_change":"2024-04-26T21:45:35.436314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:45:45.121445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa3625e2-a5b4-4ef3-82bb-4bfc8dd6961d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6801","nonce":2200991001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6803","nonce":2200991001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6813","nonce":2200991001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2200991001},{"type":"v1","addr":"172.21.15.106:6808","nonce":2200991001}]},"public_addr":"172.21.15.106:6801/2200991001","cluster_addr":"172.21.15.106:6803/2200991001","heartbeat_back_addr":"172.21.15.106:6813/2200991001","heartbeat_front_addr":"172.21.15.106:6808/2200991001","state":["exists","up"]},{"osd":1,"uuid":"2434bd98-00f9-476b-9fc9-1d32817f20a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6805","nonce":1887774310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6807","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6809","nonce":1887774310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6815","nonce":1887774310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1887774310},{"type":"v1","addr":"172.21.15.106:6812","nonce":1887774310}]},"public_addr":"172.21.15.106:6805/1887774310","cluster_addr":"172.21.15.106:6809/1887774310","heartbeat_back_addr":"172.21.15.106:6815/1887774310","heartbeat_front_addr":"172.21.15.106:6812/1887774310","state":["exists","up"]},{"osd":2,"uuid":"7232f99a-d698-4033-8f17-49f18b27bbdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6825","nonce":323734454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6827","nonce":323734454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6831","nonce":323734454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":323734454},{"type":"v1","addr":"172.21.15.106:6829","nonce":323734454}]},"public_addr":"172.21.15.106:6825/323734454","cluster_addr":"172.21.15.106:6827/323734454","heartbeat_back_addr":"172.21.15.106:6831/323734454","heartbeat_front_addr":"172.21.15.106:6829/323734454","state":["exists","up"]},{"osd":3,"uuid":"4dcbdf81-a9a2-4a82-9270-7ef5048f8cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6817","nonce":56943883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6819","nonce":56943883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6823","nonce":56943883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":56943883},{"type":"v1","addr":"172.21.15.106:6821","nonce":56943883}]},"public_addr":"172.21.15.106:6817/56943883","cluster_addr":"172.21.15.106:6819/56943883","heartbeat_back_addr":"172.21.15.106:6823/56943883","heartbeat_front_addr":"172.21.15.106:6821/56943883","state":["exists","up"]},{"osd":4,"uuid":"2cdc59f0-3c29-45d4-a2d9-9a3ba525b456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6801","nonce":3289538728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6803","nonce":3289538728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6807","nonce":3289538728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3289538728},{"type":"v1","addr":"172.21.15.204:6805","nonce":3289538728}]},"public_addr":"172.21.15.204:6801/3289538728","cluster_addr":"172.21.15.204:6803/3289538728","heartbeat_back_addr":"172.21.15.204:6807/3289538728","heartbeat_front_addr":"172.21.15.204:6805/3289538728","state":["exists","up"]},{"osd":5,"uuid":"999a8669-c8b6-4599-811c-4531d20d4c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6817","nonce":885497049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6819","nonce":885497049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6826","nonce":885497049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":885497049},{"type":"v1","addr":"172.21.15.204:6822","nonce":885497049}]},"public_addr":"172.21.15.204:6817/885497049","cluster_addr":"172.21.15.204:6819/885497049","heartbeat_back_addr":"172.21.15.204:6826/885497049","heartbeat_front_addr":"172.21.15.204:6822/885497049","state":["exists","up"]},{"osd":6,"uuid":"f1bc9749-7a5c-4612-9b9b-485f6a8fdeef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6823","nonce":3330975067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6825","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6827","nonce":3330975067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6831","nonce":3330975067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3330975067},{"type":"v1","addr":"172.21.15.204:6829","nonce":3330975067}]},"public_addr":"172.21.15.204:6823/3330975067","cluster_addr":"172.21.15.204:6827/3330975067","heartbeat_back_addr":"172.21.15.204:6831/3330975067","heartbeat_front_addr":"172.21.15.204:6829/3330975067","state":["exists","up"]},{"osd":7,"uuid":"f32beea4-f84e-49d6-9485-5d68f547e530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6809","nonce":2746439320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6811","nonce":2746439320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6815","nonce":2746439320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2746439320},{"type":"v1","addr":"172.21.15.204:6813","nonce":2746439320}]},"public_addr":"172.21.15.204:6809/2746439320","cluster_addr":"172.21.15.204:6811/2746439320","heartbeat_back_addr":"172.21.15.204:6815/2746439320","heartbeat_front_addr":"172.21.15.204:6813/2746439320","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.878144+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.969748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.874330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.585866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.789204+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.870569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.803188+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:40.401303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/1457746115":"2024-04-27T21:50:37.033347+0000","172.21.15.106:6836/1457746115":"2024-04-27T21:50:37.033347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 msgr2=0x7fa7c406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 0x7fa7c406ef60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0034000 tx=0x7fa7d002c000 comp rx=0 tx=0).stop 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 msgr2=0x7fa7e815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e9b0 tx=0x7fa7d800ee70 comp rx=0 tx=0).stop 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 shutdown_connections 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e8160930 0x7fa7e80c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa7c406caa0 0x7fa7c406ef60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8146680 0x7fa7e8073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 --2- 172.21.15.106:0/3576221828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7e80d0e70 0x7fa7e815bff0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:44.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 >> 172.21.15.106:0/3576221828 conn(0x7fa7e806c740 msgr2=0x7fa7e8141750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:44.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 shutdown_connections 2024-04-26T21:50:44.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:44.286+0000 7fa7e7200640 1 -- 172.21.15.106:0/3576221828 wait complete. 2024-04-26T21:50:45.306 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T21:50:45.307 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:45.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.496+0000 7f8680c00640 1 Processor -- start 2024-04-26T21:50:45.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.496+0000 7f8680c00640 1 -- start start 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f8680c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f867c143850 0x7f867c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f8680c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f867c0cb8b0 0x7f867c0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f8680c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c06df60 con 0x7f867c143470 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f8680c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f867c146ba0 con 0x7f867c143850 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f8680c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f867c06e200 con 0x7f867c0cb8b0 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.497+0000 7f867aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f867c143850 0x7f867c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f867c143470 0x7f867c0cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34694/0 (socket says 172.21.15.106:34694) 2024-04-26T21:50:45.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867aa00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f867c143850 0x7f867c06fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48814/0 (socket says 172.21.15.106:48814) 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867b400640 1 -- 172.21.15.106:0/440383410 learned_addr learned my addr 172.21.15.106:0/440383410 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867be00640 1 --2- 172.21.15.106:0/440383410 >> v2:172.21.15.204:3300/0 conn(0x7f867c0cb8b0 0x7f867c0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867aa00640 1 -- 172.21.15.106:0/440383410 >> v2:172.21.15.204:3300/0 conn(0x7f867c0cb8b0 msgr2=0x7f867c0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867aa00640 1 --2- 172.21.15.106:0/440383410 >> v2:172.21.15.204:3300/0 conn(0x7f867c0cb8b0 0x7f867c0723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867aa00640 1 -- 172.21.15.106:0/440383410 >> v1:172.21.15.106:6789/0 conn(0x7f867c143470 legacy=0x7f867c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.498+0000 7f867aa00640 1 -- 172.21.15.106:0/440383410 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c06e4e0 con 0x7f867c143850 2024-04-26T21:50:45.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.499+0000 7f867aa00640 1 --2- 172.21.15.106:0/440383410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143850 0x7f867c06fb10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f867c15d630 tx=0x7f866802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e91311722189ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.499+0000 7f867a000640 1 -- 172.21.15.106:0/440383410 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668037070 con 0x7f867c143850 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.499+0000 7f867a000640 1 -- 172.21.15.106:0/440383410 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668032440 con 0x7f867c143850 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.499+0000 7f867a000640 1 -- 172.21.15.106:0/440383410 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866802ae60 con 0x7f867c143850 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 -- 172.21.15.106:0/440383410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143850 msgr2=0x7f867c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 --2- 172.21.15.106:0/440383410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143850 0x7f867c06fb10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f867c15d630 tx=0x7f866802a110 comp rx=0 tx=0).stop 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 -- 172.21.15.106:0/440383410 shutdown_connections 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 --2- 172.21.15.106:0/440383410 >> v2:172.21.15.204:3300/0 conn(0x7f867c0cb8b0 0x7f867c0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 --2- 172.21.15.106:0/440383410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143850 0x7f867c06fb10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 -- 172.21.15.106:0/440383410 >> 172.21.15.106:0/440383410 conn(0x7f867c0d0b50 msgr2=0x7f867c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 -- 172.21.15.106:0/440383410 shutdown_connections 2024-04-26T21:50:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.500+0000 7f8680c00640 1 -- 172.21.15.106:0/440383410 wait complete. 2024-04-26T21:50:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.502+0000 7f8680c00640 1 Processor -- start 2024-04-26T21:50:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.502+0000 7f8680c00640 1 -- start start 2024-04-26T21:50:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c0cb8b0 0x7f867c13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 0x7f867c13e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867c143850 0x7f867c0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0cbd00 con 0x7f867c143470 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f867c070180 con 0x7f867c0cb8b0 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f8680c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f867c0cbe80 con 0x7f867c143850 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 0x7f867c13e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 0x7f867c13e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48818/0 (socket says 172.21.15.106:48818) 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 -- 172.21.15.106:0/1769360975 learned_addr learned my addr 172.21.15.106:0/1769360975 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 -- 172.21.15.106:0/1769360975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867c143850 msgr2=0x7f867c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867c143850 0x7f867c0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 -- 172.21.15.106:0/1769360975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c0cb8b0 msgr2=0x7f867c13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c0cb8b0 0x7f867c13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.503+0000 7f867aa00640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c1f8900 con 0x7f867c143470 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f867aa00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 0x7f867c13e400 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8668009890 tx=0x7f86680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668037040 con 0x7f867c143470 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668042070 con 0x7f867c143470 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c1f8ba0 con 0x7f867c143470 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668030a50 con 0x7f867c143470 2024-04-26T21:50:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.504+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c1f9160 con 0x7f867c143470 2024-04-26T21:50:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.505+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f867c06d1f0 con 0x7f867c143470 2024-04-26T21:50:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.506+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f8668030bf0 con 0x7f867c143470 2024-04-26T21:50:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f8673400640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06caa0 0x7f865c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f86680ac6b0 con 0x7f867c143470 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f867b400640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06caa0 0x7f865c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f8673400640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f865c072960 0x7f865c074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f866800af30 con 0x7f865c072960 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.507+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f86680b3050 con 0x7f867c143470 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.508+0000 7f867be00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f865c072960 0x7f865c074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.508+0000 7f867b400640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06caa0 0x7f865c06ef60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f866c000f60 tx=0x7f866c007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.508+0000 7f867be00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f865c072960 0x7f865c074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.509+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== osd.0 v2:172.21.15.106:6800/2200991001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f866800af30 con 0x7f865c072960 2024-04-26T21:50:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.523+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 --> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f867c0693c0 con 0x7f865c072960 2024-04-26T21:50:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8673400640 1 -- 172.21.15.106:0/1769360975 <== osd.0 v2:172.21.15.106:6800/2200991001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f867c0693c0 con 0x7f865c072960 2024-04-26T21:50:45.525 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:45.525 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:45.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f865c072960 msgr2=0x7f865c074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6800/2200991001,v1:172.21.15.106:6801/2200991001] conn(0x7f865c072960 0x7f865c074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06caa0 msgr2=0x7f865c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f865c06caa0 0x7f865c06ef60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f866c000f60 tx=0x7f866c007a80 comp rx=0 tx=0).stop 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 msgr2=0x7f867c13e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f8680c00640 1 --2- 172.21.15.106:0/1769360975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f867c143470 0x7f867c13e400 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8668009890 tx=0x7f86680046c0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.524+0000 7f867be00640 1 -- 172.21.15.106:0/1769360975 reap_dead start 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.525+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 shutdown_connections 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.525+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 >> 172.21.15.106:0/1769360975 conn(0x7f867c0d0b50 msgr2=0x7f867c15c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.525+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 shutdown_connections 2024-04-26T21:50:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.525+0000 7f8680c00640 1 -- 172.21.15.106:0/1769360975 wait complete. 2024-04-26T21:50:45.541 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T21:50:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.717+0000 7f9375c00640 1 Processor -- start 2024-04-26T21:50:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.717+0000 7f9375c00640 1 -- start start 2024-04-26T21:50:45.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937015f1b0 0x7f93700d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9370162b00 0x7f9370162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 0x7f93700cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9370163910 con 0x7f9370162b00 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f93700d3f50 con 0x7f9370163480 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93700d36b0 con 0x7f937015f1b0 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9370162b00 0x7f9370162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9370162b00 0x7f9370162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48822/0 (socket says 172.21.15.106:48822) 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 -- 172.21.15.106:0/3854099564 learned_addr learned my addr 172.21.15.106:0/3854099564 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375200640 1 --2- 172.21.15.106:0/3854099564 >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 0x7f93700cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936fe00640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937015f1b0 0x7f93700d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 -- 172.21.15.106:0/3854099564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937015f1b0 msgr2=0x7f93700d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937015f1b0 0x7f93700d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 -- 172.21.15.106:0/3854099564 >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 msgr2=0x7f93700cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 --2- 172.21.15.106:0/3854099564 >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 0x7f93700cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f936f400640 1 -- 172.21.15.106:0/3854099564 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93700d3990 con 0x7f9370162b00 2024-04-26T21:50:45.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.719+0000 7f9375200640 1 --2- 172.21.15.106:0/3854099564 >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 0x7f93700cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.720+0000 7f936f400640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370162b00 0x7f9370162f40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f93700c8770 tx=0x7f935c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22dfa33e8566786 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.721+0000 7f936ea00640 1 -- 172.21.15.106:0/3854099564 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c037070 con 0x7f9370162b00 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.721+0000 7f936ea00640 1 -- 172.21.15.106:0/3854099564 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c032440 con 0x7f9370162b00 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.721+0000 7f936ea00640 1 -- 172.21.15.106:0/3854099564 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c032810 con 0x7f9370162b00 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.721+0000 7f9375c00640 1 -- 172.21.15.106:0/3854099564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370162b00 msgr2=0x7f9370162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.721+0000 7f9375c00640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370162b00 0x7f9370162f40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f93700c8770 tx=0x7f935c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 -- 172.21.15.106:0/3854099564 shutdown_connections 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 --2- 172.21.15.106:0/3854099564 >> v2:172.21.15.204:3300/0 conn(0x7f9370163480 0x7f93700cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9370162b00 0x7f9370162f40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 --2- 172.21.15.106:0/3854099564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937015f1b0 0x7f93700d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 -- 172.21.15.106:0/3854099564 >> 172.21.15.106:0/3854099564 conn(0x7f937009e230 msgr2=0x7f93700d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 -- 172.21.15.106:0/3854099564 shutdown_connections 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.722+0000 7f9375c00640 1 -- 172.21.15.106:0/3854099564 wait complete. 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.723+0000 7f9375c00640 1 Processor -- start 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.723+0000 7f9375c00640 1 -- start start 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370162b00 0x7f93701f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370163480 0x7f937014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93700d33a0 con 0x7f937015f1b0 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93700b7d90 con 0x7f9370163480 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.724+0000 7f9375c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93700d0010 con 0x7f9370162b00 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48836/0 (socket says 172.21.15.106:48836) 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 -- 172.21.15.106:0/1729874950 learned_addr learned my addr 172.21.15.106:0/1729874950 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 -- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370162b00 msgr2=0x7f93701f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370162b00 0x7f93701f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 -- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370163480 msgr2=0x7f937014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370163480 0x7f937014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 -- 172.21.15.106:0/1729874950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937014be80 con 0x7f937015f1b0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f936fe00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f936000c7f0 tx=0x7f936000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.725+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360015070 con 0x7f937015f1b0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.726+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9360004220 con 0x7f937015f1b0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.726+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360003610 con 0x7f937015f1b0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.726+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9370160bc0 con 0x7f937015f1b0 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.726+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9370161180 con 0x7f937015f1b0 2024-04-26T21:50:45.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.727+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93700693c0 con 0x7f937015f1b0 2024-04-26T21:50:45.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.728+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f93600039c0 con 0x7f937015f1b0 2024-04-26T21:50:45.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.729+0000 7f9367e00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 0x7f935006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.729+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f936008bda0 con 0x7f937015f1b0 2024-04-26T21:50:45.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.730+0000 7f936f400640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 0x7f935006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.730+0000 7f936f400640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 0x7f935006ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f935c034000 tx=0x7f935c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.734+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f936005a0b0 con 0x7f937015f1b0 2024-04-26T21:50:45.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.949+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f93700c8310 con 0x7f935006caa0 2024-04-26T21:50:45.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.949+0000 7f9367e00640 1 -- 172.21.15.106:0/1729874950 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f935c032670 con 0x7f935006caa0 2024-04-26T21:50:45.950 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T21:50:45.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 msgr2=0x7f935006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 0x7f935006ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f935c034000 tx=0x7f935c02c000 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 msgr2=0x7f93701605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f936000c7f0 tx=0x7f936000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 shutdown_connections 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9370163480 0x7f937014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f935006caa0 0x7f935006ef60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9370162b00 0x7f93701f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 --2- 172.21.15.106:0/1729874950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f937015f1b0 0x7f93701605e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 >> 172.21.15.106:0/1729874950 conn(0x7f937009e230 msgr2=0x7f93700d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 shutdown_connections 2024-04-26T21:50:45.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:45.952+0000 7f9375c00640 1 -- 172.21.15.106:0/1729874950 wait complete. 2024-04-26T21:50:45.971 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T21:50:45.971 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:46.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.158+0000 7f0cce400640 1 Processor -- start 2024-04-26T21:50:46.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.158+0000 7f0cce400640 1 -- start start 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0cce400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 0x7f0cc80d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0cce400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cc80d0e70 0x7f0cc80d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0cce400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc80d2e60 con 0x7f0cc80d3f00 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0cce400640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d3a70 con 0x7f0cc80d0e70 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0cce400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d3140 con 0x7f0cc815f1c0 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0ccda00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cc80d0e70 0x7f0cc80d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0ccda00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cc80d0e70 0x7f0cc80d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48838/0 (socket says 172.21.15.106:48838) 2024-04-26T21:50:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.159+0000 7f0ccda00640 1 -- 172.21.15.106:0/2203568476 learned_addr learned my addr 172.21.15.106:0/2203568476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccda00640 1 -- 172.21.15.106:0/2203568476 >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 msgr2=0x7f0cc80d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccd000640 1 --2- 172.21.15.106:0/2203568476 >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 0x7f0cc80d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccda00640 1 --2- 172.21.15.106:0/2203568476 >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 0x7f0cc80d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccda00640 1 -- 172.21.15.106:0/2203568476 >> v1:172.21.15.204:6789/0 conn(0x7f0cc80d3f00 legacy=0x7f0cc80d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccda00640 1 -- 172.21.15.106:0/2203568476 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80d3420 con 0x7f0cc80d0e70 2024-04-26T21:50:46.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.160+0000 7f0ccd000640 1 --2- 172.21.15.106:0/2203568476 >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 0x7f0cc80d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:46.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.161+0000 7f0ccda00640 1 --2- 172.21.15.106:0/2203568476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc80d1f70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d3bf0 tx=0x7f0cc002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4e21f0dff7941b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.161+0000 7f0cc7400640 1 -- 172.21.15.106:0/2203568476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0037070 con 0x7f0cc80d0e70 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.161+0000 7f0cc7400640 1 -- 172.21.15.106:0/2203568476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc0032440 con 0x7f0cc80d0e70 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.161+0000 7f0cc7400640 1 -- 172.21.15.106:0/2203568476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0032810 con 0x7f0cc80d0e70 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 -- 172.21.15.106:0/2203568476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 msgr2=0x7f0cc80d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 --2- 172.21.15.106:0/2203568476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc80d1f70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d3bf0 tx=0x7f0cc002a210 comp rx=0 tx=0).stop 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 -- 172.21.15.106:0/2203568476 shutdown_connections 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 --2- 172.21.15.106:0/2203568476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc80d1f70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 --2- 172.21.15.106:0/2203568476 >> v2:172.21.15.204:3300/0 conn(0x7f0cc815f1c0 0x7f0cc80d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 -- 172.21.15.106:0/2203568476 >> 172.21.15.106:0/2203568476 conn(0x7f0cc809e340 msgr2=0x7f0cc80d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 -- 172.21.15.106:0/2203568476 shutdown_connections 2024-04-26T21:50:46.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.162+0000 7f0cce400640 1 -- 172.21.15.106:0/2203568476 wait complete. 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.163+0000 7f0cce400640 1 Processor -- start 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 -- start start 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80d3f00 0x7f0cc815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc815f1c0 0x7f0cc8153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc815f720 con 0x7f0cc80d0e70 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc815f5a0 con 0x7f0cc80d3f00 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.164+0000 7f0cce400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80b48a0 con 0x7f0cc815f1c0 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48850/0 (socket says 172.21.15.106:48850) 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 -- 172.21.15.106:0/3716746853 learned_addr learned my addr 172.21.15.106:0/3716746853 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 -- 172.21.15.106:0/3716746853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc815f1c0 msgr2=0x7f0cc8153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc815f1c0 0x7f0cc8153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 -- 172.21.15.106:0/3716746853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80d3f00 msgr2=0x7f0cc815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80d3f00 0x7f0cc815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc81601f0 con 0x7f0cc80d0e70 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.165+0000 7f0ccd000640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc815f8a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800ce30 tx=0x7f0cb800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.166+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800f890 con 0x7f0cc80d0e70 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.166+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb800fa30 con 0x7f0cc80d0e70 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.166+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800fe00 con 0x7f0cc80d0e70 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.166+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81f85e0 con 0x7f0cc80d0e70 2024-04-26T21:50:46.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.166+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81f8ba0 con 0x7f0cc80d0e70 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.167+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cc80693c0 con 0x7f0cc80d0e70 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.168+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0cb8018400 con 0x7f0cc80d0e70 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc4c00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ca406caa0 0x7f0ca406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f0cb808ca70 con 0x7f0cc80d0e70 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc4c00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7f0ca4072960 0x7f0ca4074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc7e00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ca406caa0 0x7f0ca406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ca4075430 con 0x7f0ca4072960 2024-04-26T21:50:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0cb80912a0 con 0x7f0cc80d0e70 2024-04-26T21:50:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.169+0000 7f0ccda00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7f0ca4072960 0x7f0ca4074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.170+0000 7f0cc7e00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ca406caa0 0x7f0ca406ef60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d0a40 tx=0x7f0cb0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.170+0000 7f0ccda00640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7f0ca4072960 0x7f0ca4074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.171+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== osd.1 v2:172.21.15.106:6804/1887774310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ca4075430 con 0x7f0ca4072960 2024-04-26T21:50:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.184+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 --> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0cc80b1e60 con 0x7f0ca4072960 2024-04-26T21:50:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cc4c00640 1 -- 172.21.15.106:0/3716746853 <== osd.1 v2:172.21.15.106:6804/1887774310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0cc80b1e60 con 0x7f0ca4072960 2024-04-26T21:50:46.186 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:46.186 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:46.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7f0ca4072960 msgr2=0x7f0ca4074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6804/1887774310,v1:172.21.15.106:6805/1887774310] conn(0x7f0ca4072960 0x7f0ca4074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ca406caa0 msgr2=0x7f0ca406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0ca406caa0 0x7f0ca406ef60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d0a40 tx=0x7f0cb0007990 comp rx=0 tx=0).stop 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 msgr2=0x7f0cc815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 --2- 172.21.15.106:0/3716746853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc80d0e70 0x7f0cc815f8a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800ce30 tx=0x7f0cb800a650 comp rx=0 tx=0).stop 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0ccda00640 1 -- 172.21.15.106:0/3716746853 reap_dead start 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 shutdown_connections 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.185+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 >> 172.21.15.106:0/3716746853 conn(0x7f0cc809e340 msgr2=0x7f0cc815d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.186+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 shutdown_connections 2024-04-26T21:50:46.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.186+0000 7f0cce400640 1 -- 172.21.15.106:0/3716746853 wait complete. 2024-04-26T21:50:46.202 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T21:50:46.303 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:50:46.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:50:46.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.395+0000 7fd577200640 1 Processor -- start 2024-04-26T21:50:46.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.395+0000 7fd577200640 1 -- start start 2024-04-26T21:50:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.396+0000 7fd577200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd578163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.396+0000 7fd577200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5780d3f50 con 0x7fd57815f1b0 2024-04-26T21:50:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.396+0000 7fd577200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5780d36b0 con 0x7fd5780b82b0 2024-04-26T21:50:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.396+0000 7fd577200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780d3990 con 0x7fd5780d5990 2024-04-26T21:50:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd575e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd57815f1b0 0x7fd578162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34710/0 (socket says 172.21.15.106:34710) 2024-04-26T21:50:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd575e00640 1 -- 172.21.15.106:0/3044364595 learned_addr learned my addr 172.21.15.106:0/3044364595 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd575400640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd578163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247821049 0 0) 0x7fd5780d3f50 con 0x7fd57815f1b0 2024-04-26T21:50:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd57815f1b0 2024-04-26T21:50:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.397+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1701521617 0 0) 0x7fd55c003520 con 0x7fd57815f1b0 2024-04-26T21:50:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 msgr2=0x7fd578163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd578163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 >> v1:172.21.15.204:6789/0 conn(0x7fd5780b82b0 legacy=0x7fd5780b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780d3c70 con 0x7fd57815f1b0 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fd568001670 con 0x7fd57815f1b0 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 >> v1:172.21.15.106:6789/0 conn(0x7fd57815f1b0 legacy=0x7fd578162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd55c0039e0 0x7fd55c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd55c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780d3990 con 0x7fd55c0039e0 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c00ba70 con 0x7fd55c004d40 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c00bc70 con 0x7fd5780d5990 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd55c0039e0 0x7fd55c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd576800640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 -- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 msgr2=0x7fd55c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575400640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd55c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd55c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 -- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd55c004d40 msgr2=0x7fd55c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575e00640 1 -- 172.21.15.106:0/3044364595 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c003520 con 0x7fd55c0039e0 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd575400640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5780d5990 0x7fd55c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.398+0000 7fd576800640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.399+0000 7fd575e00640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd55c0039e0 0x7fd55c003e00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd568001eb0 tx=0x7fd55c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf3a9a0b83f8da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568007d30 con 0x7fd55c0039e0 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5680058d0 con 0x7fd55c0039e0 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd574a00640 1 -- 172.21.15.106:0/3044364595 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568005ce0 con 0x7fd55c0039e0 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 -- 172.21.15.106:0/3044364595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd55c0039e0 msgr2=0x7fd55c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 --2- 172.21.15.106:0/3044364595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd55c0039e0 0x7fd55c003e00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd568001eb0 tx=0x7fd55c005830 comp rx=0 tx=0).stop 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd576800640 1 -- 172.21.15.106:0/3044364595 reap_dead start 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 -- 172.21.15.106:0/3044364595 shutdown_connections 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 -- 172.21.15.106:0/3044364595 >> 172.21.15.106:0/3044364595 conn(0x7fd57809e230 msgr2=0x7fd5780d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 -- 172.21.15.106:0/3044364595 shutdown_connections 2024-04-26T21:50:46.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.400+0000 7fd577200640 1 -- 172.21.15.106:0/3044364595 wait complete. 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.402+0000 7fd577200640 1 Processor -- start 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.402+0000 7fd577200640 1 -- start start 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd55c0039e0 0x7fd578149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd57814a680 0x7fd5780ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd578163210 con 0x7fd57815a2f0 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780d33a0 con 0x7fd57814a680 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd577200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd578163390 con 0x7fd55c0039e0 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd575400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd575400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48872/0 (socket says 172.21.15.106:48872) 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd575400640 1 -- 172.21.15.106:0/896168295 learned_addr learned my addr 172.21.15.106:0/896168295 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd575e00640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd55c0039e0 0x7fd578149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.403+0000 7fd576800640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd57814a680 0x7fd5780ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 -- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd55c0039e0 msgr2=0x7fd578149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd55c0039e0 0x7fd578149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 -- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd57814a680 msgr2=0x7fd5780ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd57814a680 0x7fd5780ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 -- 172.21.15.106:0/896168295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780cd890 con 0x7fd57815a2f0 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd575400640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd5640108b0 tx=0x7fd564010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56400f040 con 0x7fd57815a2f0 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56400eb60 con 0x7fd57815a2f0 2024-04-26T21:50:46.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56401c400 con 0x7fd57815a2f0 2024-04-26T21:50:46.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5780cdb70 con 0x7fd57815a2f0 2024-04-26T21:50:46.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.404+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5780ce130 con 0x7fd57815a2f0 2024-04-26T21:50:46.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.405+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5780693c0 con 0x7fd57815a2f0 2024-04-26T21:50:46.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.406+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fd56401c5a0 con 0x7fd57815a2f0 2024-04-26T21:50:46.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.408+0000 7fd56e000640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 0x7fd55806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.408+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd56408c1f0 con 0x7fd57815a2f0 2024-04-26T21:50:46.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.408+0000 7fd575e00640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 0x7fd55806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.409+0000 7fd575e00640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 0x7fd55806ef60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd568033000 tx=0x7fd568035040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.412+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd564018030 con 0x7fd57815a2f0 2024-04-26T21:50:46.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.613+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd57808d090 con 0x7fd55806caa0 2024-04-26T21:50:46.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.614+0000 7fd56e000640 1 -- 172.21.15.106:0/896168295 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd57808d090 con 0x7fd55806caa0 2024-04-26T21:50:46.615 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.616+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 msgr2=0x7fd55806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.616+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 0x7fd55806ef60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd568033000 tx=0x7fd568035040 comp rx=0 tx=0).stop 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.616+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 msgr2=0x7fd57814a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd5640108b0 tx=0x7fd564010d70 comp rx=0 tx=0).stop 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 shutdown_connections 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd57814a680 0x7fd5780ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd57815a2f0 0x7fd57814a140 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fd55806caa0 0x7fd55806ef60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 --2- 172.21.15.106:0/896168295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd55c0039e0 0x7fd578149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 >> 172.21.15.106:0/896168295 conn(0x7fd57809e230 msgr2=0x7fd57806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 shutdown_connections 2024-04-26T21:50:46.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.617+0000 7fd577200640 1 -- 172.21.15.106:0/896168295 wait complete. 2024-04-26T21:50:46.636 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T21:50:46.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.829+0000 7f2c67200640 1 Processor -- start 2024-04-26T21:50:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.829+0000 7f2c67200640 1 -- start start 2024-04-26T21:50:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c6815f1c0 0x7f2c680d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2c680d3f00 0x7f2c680d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c680d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2c680d1130 con 0x7f2c6815f1c0 2024-04-26T21:50:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2c680d3a70 con 0x7f2c680d3f00 2024-04-26T21:50:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c67200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c680d2e60 con 0x7f2c680d0760 2024-04-26T21:50:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c65e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c6815f1c0 0x7f2c680d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c65e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c6815f1c0 0x7f2c680d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48886/0 (socket says 172.21.15.106:48886) 2024-04-26T21:50:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.831+0000 7f2c65e00640 1 -- 172.21.15.106:0/1383993307 learned_addr learned my addr 172.21.15.106:0/1383993307 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:46.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c66800640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c680d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c5fe00640 1 --2- 172.21.15.106:0/1383993307 >> v2:172.21.15.204:3300/0 conn(0x7f2c680d3f00 0x7f2c680d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c65e00640 1 -- 172.21.15.106:0/1383993307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 msgr2=0x7f2c680d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c65e00640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c680d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c65e00640 1 -- 172.21.15.106:0/1383993307 >> v2:172.21.15.204:3300/0 conn(0x7f2c680d3f00 msgr2=0x7f2c680d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c65e00640 1 --2- 172.21.15.106:0/1383993307 >> v2:172.21.15.204:3300/0 conn(0x7f2c680d3f00 0x7f2c680d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c65e00640 1 -- 172.21.15.106:0/1383993307 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c680d3140 con 0x7f2c6815f1c0 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.832+0000 7f2c66800640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c680d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.833+0000 7f2c65e00640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c680d1a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2c680d61d0 tx=0x7f2c5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e43d5efd63c2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.833+0000 7f2c65400640 1 -- 172.21.15.106:0/1383993307 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58037070 con 0x7f2c6815f1c0 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.833+0000 7f2c65400640 1 -- 172.21.15.106:0/1383993307 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c58032440 con 0x7f2c6815f1c0 2024-04-26T21:50:46.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.833+0000 7f2c65400640 1 -- 172.21.15.106:0/1383993307 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58032810 con 0x7f2c6815f1c0 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 -- 172.21.15.106:0/1383993307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 msgr2=0x7f2c680d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c680d1a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2c680d61d0 tx=0x7f2c5802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 -- 172.21.15.106:0/1383993307 shutdown_connections 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c680d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 --2- 172.21.15.106:0/1383993307 >> v2:172.21.15.204:3300/0 conn(0x7f2c680d3f00 0x7f2c680d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 --2- 172.21.15.106:0/1383993307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c680d1a30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 -- 172.21.15.106:0/1383993307 >> 172.21.15.106:0/1383993307 conn(0x7f2c6809e340 msgr2=0x7f2c680d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 -- 172.21.15.106:0/1383993307 shutdown_connections 2024-04-26T21:50:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.834+0000 7f2c67200640 1 -- 172.21.15.106:0/1383993307 wait complete. 2024-04-26T21:50:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.836+0000 7f2c67200640 1 Processor -- start 2024-04-26T21:50:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.836+0000 7f2c67200640 1 -- start start 2024-04-26T21:50:46.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c6815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c680d3f00 0x7f2c681522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c681537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6815f5a0 con 0x7f2c6815f1c0 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c680d3700 con 0x7f2c680d3f00 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c67200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6815f720 con 0x7f2c680d0760 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c66800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c681537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c66800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c681537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48894/0 (socket says 172.21.15.106:48894) 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.837+0000 7f2c66800640 1 -- 172.21.15.106:0/2352463796 learned_addr learned my addr 172.21.15.106:0/2352463796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c5fe00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c680d3f00 0x7f2c681522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c65e00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c6815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 -- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 msgr2=0x7f2c6815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c680d0760 0x7f2c6815fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 -- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c680d3f00 msgr2=0x7f2c681522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c680d3f00 0x7f2c681522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c681527e0 con 0x7f2c6815f1c0 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c5fe00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c680d3f00 0x7f2c681522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c66800640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c681537e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000cd10 tx=0x7f2c6000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000f680 con 0x7f2c6815f1c0 2024-04-26T21:50:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.838+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6000f840 con 0x7f2c6815f1c0 2024-04-26T21:50:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.839+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000fc30 con 0x7f2c6815f1c0 2024-04-26T21:50:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.839+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c68160470 con 0x7f2c6815f1c0 2024-04-26T21:50:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.839+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c68160a30 con 0x7f2c6815f1c0 2024-04-26T21:50:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.839+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c680693c0 con 0x7f2c6815f1c0 2024-04-26T21:50:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.841+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2c600213f0 con 0x7f2c6815f1c0 2024-04-26T21:50:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.841+0000 7f2c5e000640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c4406caa0 0x7f2c4406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f2c6008ca80 con 0x7f2c6815f1c0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c65e00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c4406caa0 0x7f2c4406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c5e000640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f2c44072960 0x7f2c44074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c6001ee80 con 0x7f2c44072960 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2c600912a0 con 0x7f2c6815f1c0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c5fe00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f2c44072960 0x7f2c44074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.842+0000 7f2c65e00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c4406caa0 0x7f2c4406ef60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c58034000 tx=0x7f2c5802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.843+0000 7f2c5fe00640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f2c44072960 0x7f2c44074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.843+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== osd.2 v2:172.21.15.106:6824/323734454 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c6001ee80 con 0x7f2c44072960 2024-04-26T21:50:46.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.857+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 --> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2c680d26e0 con 0x7f2c44072960 2024-04-26T21:50:46.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c5e000640 1 -- 172.21.15.106:0/2352463796 <== osd.2 v2:172.21.15.106:6824/323734454 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2c680d26e0 con 0x7f2c44072960 2024-04-26T21:50:46.859 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:46.860 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:46.860 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:46.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f2c44072960 msgr2=0x7f2c44074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6824/323734454,v1:172.21.15.106:6825/323734454] conn(0x7f2c44072960 0x7f2c44074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c4406caa0 msgr2=0x7f2c4406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c4406caa0 0x7f2c4406ef60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c58034000 tx=0x7f2c5802e000 comp rx=0 tx=0).stop 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 msgr2=0x7f2c681537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 --2- 172.21.15.106:0/2352463796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c6815f1c0 0x7f2c681537e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000cd10 tx=0x7f2c6000a650 comp rx=0 tx=0).stop 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c66800640 1 -- 172.21.15.106:0/2352463796 reap_dead start 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 shutdown_connections 2024-04-26T21:50:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 >> 172.21.15.106:0/2352463796 conn(0x7f2c6809e340 msgr2=0x7f2c680b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 shutdown_connections 2024-04-26T21:50:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:46.858+0000 7f2c67200640 1 -- 172.21.15.106:0/2352463796 wait complete. 2024-04-26T21:50:46.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T21:50:47.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.071+0000 7f2cb8000640 1 Processor -- start 2024-04-26T21:50:47.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.071+0000 7f2cb8000640 1 -- start start 2024-04-26T21:50:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb8000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2cb006e130 0x7f2cb00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb8000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00ae370 con 0x7f2cb00737c0 2024-04-26T21:50:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb8000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00ae5f0 con 0x7f2cb006fb50 2024-04-26T21:50:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb8000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb0072010 con 0x7f2cb006e130 2024-04-26T21:50:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb6000640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2cb00737c0 0x7f2cb006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34720/0 (socket says 172.21.15.106:34720) 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb6000640 1 -- 172.21.15.106:0/2727491309 learned_addr learned my addr 172.21.15.106:0/2727491309 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.073+0000 7f2cb5600640 1 --2- 172.21.15.106:0/2727491309 >> v2:172.21.15.204:3300/0 conn(0x7f2cb006e130 0x7f2cb00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458278057 0 0) 0x7f2ca8001490 con 0x7f2cb00737c0 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c98005730 con 0x7f2cb00737c0 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 34414892 0 0) 0x7f2c98005730 con 0x7f2cb00737c0 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 >> v2:172.21.15.204:3300/0 conn(0x7f2cb006e130 msgr2=0x7f2cb00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 --2- 172.21.15.106:0/2727491309 >> v2:172.21.15.204:3300/0 conn(0x7f2cb006e130 0x7f2cb00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 >> v1:172.21.15.204:6789/0 conn(0x7f2cb006fb50 legacy=0x7f2cb00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00722f0 con 0x7f2cb00737c0 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f2ca8001bc0 con 0x7f2cb00737c0 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 >> v1:172.21.15.106:6789/0 conn(0x7f2cb00737c0 legacy=0x7f2cb006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c98005bc0 0x7f2c98005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98006a10 0x7f2c98006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 0x7f2c9800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0072010 con 0x7f2c980073d0 2024-04-26T21:50:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9800e270 con 0x7f2c98006a10 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9800e470 con 0x7f2c98005bc0 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.074+0000 7f2cb6000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 0x7f2c9800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb5600640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c98005bc0 0x7f2c98005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb6000640 1 -- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c98005bc0 msgr2=0x7f2c98005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb6000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c98005bc0 0x7f2c98005fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb4c00640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98006a10 0x7f2c98006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb6000640 1 -- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98006a10 msgr2=0x7f2c98006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb6000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98006a10 0x7f2c98006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.075+0000 7f2cb6000640 1 -- 172.21.15.106:0/2727491309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c98005730 con 0x7f2c980073d0 2024-04-26T21:50:47.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.076+0000 7f2cb6000640 1 -- 172.21.15.106:0/2727491309 reap_dead start 2024-04-26T21:50:47.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.076+0000 7f2cb6000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 0x7f2c9800dbd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800aa20 tx=0x7f2c98007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93ffc152a8057de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.076+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800a640 con 0x7f2c980073d0 2024-04-26T21:50:47.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.076+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca802f070 con 0x7f2c980073d0 2024-04-26T21:50:47.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.076+0000 7f2cafe00640 1 -- 172.21.15.106:0/2727491309 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8034470 con 0x7f2c980073d0 2024-04-26T21:50:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 -- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 msgr2=0x7f2c9800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 0x7f2c9800dbd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800aa20 tx=0x7f2c98007a40 comp rx=0 tx=0).stop 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 -- 172.21.15.106:0/2727491309 shutdown_connections 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 --2- 172.21.15.106:0/2727491309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c980073d0 0x7f2c9800dbd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 -- 172.21.15.106:0/2727491309 >> 172.21.15.106:0/2727491309 conn(0x7f2cb00b2d50 msgr2=0x7f2cb00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 -- 172.21.15.106:0/2727491309 shutdown_connections 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.077+0000 7f2cb8000640 1 -- 172.21.15.106:0/2727491309 wait complete. 2024-04-26T21:50:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.079+0000 7f2cb8000640 1 Processor -- start 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.079+0000 7f2cb8000640 1 -- start start 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 0x7f2cb014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 0x7f2cb009e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0072b90 con 0x7f2cb009bac0 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00728b0 con 0x7f2c980073d0 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00b4810 con 0x7f2cb009dbb0 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48922/0 (socket says 172.21.15.106:48922) 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.080+0000 7f2cb6000640 1 -- 172.21.15.106:0/1691070353 learned_addr learned my addr 172.21.15.106:0/1691070353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb4c00640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 0x7f2cb009e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb5600640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 0x7f2cb014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 -- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 msgr2=0x7f2cb009e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 0x7f2cb009e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 -- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 msgr2=0x7f2cb014b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 0x7f2cb014b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 -- 172.21.15.106:0/1691070353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb009c810 con 0x7f2cb009bac0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb5600640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 0x7f2cb014b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb4c00640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 0x7f2cb009e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb6000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2ca802b6e0 tx=0x7f2cb00724c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca802be40 con 0x7f2cb009bac0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca8001490 con 0x7f2cb009bac0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb009caf0 con 0x7f2cb009bac0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.081+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8004d60 con 0x7f2cb009bac0 2024-04-26T21:50:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.082+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb009d0b0 con 0x7f2cb009bac0 2024-04-26T21:50:47.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.083+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb009d3e0 con 0x7f2cb009bac0 2024-04-26T21:50:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.084+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2ca8034470 con 0x7f2cb009bac0 2024-04-26T21:50:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.085+0000 7f2cad600640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 0x7f2c9006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.085+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f2ca80abb10 con 0x7f2cb009bac0 2024-04-26T21:50:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.085+0000 7f2cb5600640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 0x7f2c9006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.086+0000 7f2cb5600640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 0x7f2c9006ef90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40099e0 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.089+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cb00ae370 con 0x7f2cb009bac0 2024-04-26T21:50:47.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.296+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f2cb0069540 con 0x7f2c9006cad0 2024-04-26T21:50:47.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.297+0000 7f2cad600640 1 -- 172.21.15.106:0/1691070353 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2cb0069540 con 0x7f2c9006cad0 2024-04-26T21:50:47.298 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T21:50:47.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.299+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 msgr2=0x7f2c9006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.299+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 0x7f2c9006ef90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40099e0 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2024-04-26T21:50:47.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.299+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 msgr2=0x7f2cb009bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.299+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2ca802b6e0 tx=0x7f2cb00724c0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 shutdown_connections 2024-04-26T21:50:47.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb009bac0 0x7f2cb009bf40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f2c9006cad0 0x7f2c9006ef90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cb009dbb0 0x7f2cb009e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 --2- 172.21.15.106:0/1691070353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c980073d0 0x7f2cb014b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 >> 172.21.15.106:0/1691070353 conn(0x7f2cb00b2d50 msgr2=0x7f2cb014d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 shutdown_connections 2024-04-26T21:50:47.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.300+0000 7f2cb8000640 1 -- 172.21.15.106:0/1691070353 wait complete. 2024-04-26T21:50:47.318 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T21:50:47.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:47.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.523+0000 7f75ce800640 1 Processor -- start 2024-04-26T21:50:47.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.524+0000 7f75ce800640 1 -- start start 2024-04-26T21:50:47.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75ce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75c80af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75ce800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f75c80afd40 0x7f75c8072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75ce800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c80ae160 con 0x7f75c815d170 2024-04-26T21:50:47.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75ce800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f75c815d550 con 0x7f75c80afd40 2024-04-26T21:50:47.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75ce800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75c806fd30 con 0x7f75c80cffe0 2024-04-26T21:50:47.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.525+0000 7f75c7400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f75c815d170 0x7f75c806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34736/0 (socket says 172.21.15.106:34736) 2024-04-26T21:50:47.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c7400640 1 -- 172.21.15.106:0/1007146143 learned_addr learned my addr 172.21.15.106:0/1007146143 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:47.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1007146143 >> v2:172.21.15.204:3300/0 conn(0x7f75c80afd40 0x7f75c8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75c80af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615709242 0 0) 0x7f75c80ae160 con 0x7f75c815d170 2024-04-26T21:50:47.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a8003520 con 0x7f75c815d170 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 230095257 0 0) 0x7f75a8003520 con 0x7f75c815d170 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 msgr2=0x7f75c80af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75c80af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 >> v2:172.21.15.204:3300/0 conn(0x7f75c80afd40 msgr2=0x7f75c8072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 --2- 172.21.15.106:0/1007146143 >> v2:172.21.15.204:3300/0 conn(0x7f75c80afd40 0x7f75c8072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c806ff80 con 0x7f75c815d170 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f75b8001670 con 0x7f75c815d170 2024-04-26T21:50:47.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.526+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 >> v1:172.21.15.106:6789/0 conn(0x7f75c815d170 legacy=0x7f75c806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75c815d550 con 0x7f75a8004d40 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75c806fd30 con 0x7f75a8003f30 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75a800b9c0 con 0x7f75c80cffe0 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7400640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 -- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 msgr2=0x7f75a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c80cffe0 0x7f75a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 -- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8003f30 msgr2=0x7f75a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c7e00640 1 -- 172.21.15.106:0/1007146143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8003520 con 0x7f75a8004d40 2024-04-26T21:50:47.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.527+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8003f30 0x7f75a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.528+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009ac0 tx=0x7f75a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a346a5c45a1747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0036070 con 0x7f75a8004d40 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c0031400 con 0x7f75a8004d40 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75c6000640 1 -- 172.21.15.106:0/1007146143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0009690 con 0x7f75a8004d40 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75ce800640 1 -- 172.21.15.106:0/1007146143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75a8004d40 msgr2=0x7f75a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75ce800640 1 --2- 172.21.15.106:0/1007146143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75a8004d40 0x7f75a80051c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009ac0 tx=0x7f75a8005830 comp rx=0 tx=0).stop 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75c7e00640 1 -- 172.21.15.106:0/1007146143 reap_dead start 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75ce800640 1 -- 172.21.15.106:0/1007146143 shutdown_connections 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.529+0000 7f75ce800640 1 -- 172.21.15.106:0/1007146143 >> 172.21.15.106:0/1007146143 conn(0x7f75c80d0c40 msgr2=0x7f75c80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.530+0000 7f75ce800640 1 -- 172.21.15.106:0/1007146143 shutdown_connections 2024-04-26T21:50:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.530+0000 7f75ce800640 1 -- 172.21.15.106:0/1007146143 wait complete. 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.531+0000 7f75ce800640 1 Processor -- start 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.532+0000 7f75ce800640 1 -- start start 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.532+0000 7f75ce800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8004d40 0x7f75c81435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.532+0000 7f75ce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c8159c20 0x7f75c815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.532+0000 7f75ce800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 0x7f75c8143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.532+0000 7f75ce800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75c80d0440 con 0x7f75c815fe30 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75ce800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75c806ea40 con 0x7f75a8004d40 2024-04-26T21:50:47.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75ce800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75c80b01d0 con 0x7f75c8159c20 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 0x7f75c8143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 0x7f75c8143ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48938/0 (socket says 172.21.15.106:48938) 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 -- 172.21.15.106:0/1266505878 learned_addr learned my addr 172.21.15.106:0/1266505878 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8004d40 0x7f75c81435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c8159c20 0x7f75c815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 -- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c8159c20 msgr2=0x7f75c815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c8159c20 0x7f75c815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 -- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8004d40 msgr2=0x7f75c81435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8004d40 0x7f75c81435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7e00640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c815e850 con 0x7f75c815fe30 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c7400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a8004d40 0x7f75c81435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.533+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c8159c20 0x7f75c815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75c7e00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 0x7f75c8143ec0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009ac0 tx=0x7f75c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0036040 con 0x7f75c815fe30 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c0041070 con 0x7f75c815fe30 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c815eb30 con 0x7f75c815fe30 2024-04-26T21:50:47.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0003c80 con 0x7f75c815fe30 2024-04-26T21:50:47.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.534+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c815f0f0 con 0x7f75c815fe30 2024-04-26T21:50:47.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.535+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75c8069650 con 0x7f75c815fe30 2024-04-26T21:50:47.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.536+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f75c0039070 con 0x7f75c815fe30 2024-04-26T21:50:47.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75bf400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f75a406caa0 0x7f75a406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f75c00ab560 con 0x7f75c815fe30 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75c7400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f75a406caa0 0x7f75a406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75bf400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7f75a4072960 0x7f75a4074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75a4075430 con 0x7f75a4072960 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.537+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f75c00b2050 con 0x7f75c815fe30 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.538+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7f75a4072960 0x7f75a4074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.538+0000 7f75c7400640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f75a406caa0 0x7f75a406ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005990 tx=0x7f75b8001500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.538+0000 7f75c6a00640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7f75a4072960 0x7f75a4074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.539+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== osd.3 v2:172.21.15.106:6816/56943883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75a4075430 con 0x7f75a4072960 2024-04-26T21:50:47.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.554+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 --> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f75c80b0550 con 0x7f75a4072960 2024-04-26T21:50:47.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.554+0000 7f75bf400640 1 -- 172.21.15.106:0/1266505878 <== osd.3 v2:172.21.15.106:6816/56943883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f75c80b0550 con 0x7f75a4072960 2024-04-26T21:50:47.555 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:47.555 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:47.556 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:47.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.554+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7f75a4072960 msgr2=0x7f75a4074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.554+0000 7f75ce800640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6816/56943883,v1:172.21.15.106:6817/56943883] conn(0x7f75a4072960 0x7f75a4074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f75a406caa0 msgr2=0x7f75a406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f75a406caa0 0x7f75a406ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005990 tx=0x7f75b8001500 comp rx=0 tx=0).stop 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 msgr2=0x7f75c8143ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 --2- 172.21.15.106:0/1266505878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75c815fe30 0x7f75c8143ec0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009ac0 tx=0x7f75c0004730 comp rx=0 tx=0).stop 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75c7e00640 1 -- 172.21.15.106:0/1266505878 reap_dead start 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 shutdown_connections 2024-04-26T21:50:47.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 >> 172.21.15.106:0/1266505878 conn(0x7f75c80d0c40 msgr2=0x7f75c80b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 shutdown_connections 2024-04-26T21:50:47.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.555+0000 7f75ce800640 1 -- 172.21.15.106:0/1266505878 wait complete. 2024-04-26T21:50:47.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T21:50:47.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.769+0000 7f932b200640 1 Processor -- start 2024-04-26T21:50:47.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.769+0000 7f932b200640 1 -- start start 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f932b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f932c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f932b200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0727f0 con 0x7f932c0d0e70 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f932b200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0b4a10 con 0x7f932c0b5660 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f932b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0b4cf0 con 0x7f932c0b5d40 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f9329e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f932c0d0e70 0x7f932c0d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34738/0 (socket says 172.21.15.106:34738) 2024-04-26T21:50:47.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f9329e00640 1 -- 172.21.15.106:0/3536250364 learned_addr learned my addr 172.21.15.106:0/3536250364 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:47.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.771+0000 7f932a800640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f932c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123524633 0 0) 0x7f932c0727f0 con 0x7f932c0d0e70 2024-04-26T21:50:47.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9310003520 con 0x7f932c0d0e70 2024-04-26T21:50:47.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1821097411 0 0) 0x7f9310003520 con 0x7f932c0d0e70 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 msgr2=0x7f932c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f932c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 >> v1:172.21.15.204:6789/0 conn(0x7f932c0b5660 legacy=0x7f932c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0b4fd0 con 0x7f932c0d0e70 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f931c001670 con 0x7f932c0d0e70 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 >> v1:172.21.15.106:6789/0 conn(0x7f932c0d0e70 legacy=0x7f932c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93100039e0 0x7f9310003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f9310004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9310004d40 0x7f93100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0b4cf0 con 0x7f9310004d40 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f931000baf0 con 0x7f93100039e0 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f931000bcf0 con 0x7f932c0b5d40 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.772+0000 7f932a800640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9310004d40 0x7f93100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f9329400640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f9310004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f9329e00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93100039e0 0x7f9310003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f932a800640 1 -- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 msgr2=0x7f9310004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f932a800640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f9310004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f932a800640 1 -- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93100039e0 msgr2=0x7f9310003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f932a800640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93100039e0 0x7f9310003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f932a800640 1 -- 172.21.15.106:0/3536250364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9310003520 con 0x7f9310004d40 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f9329400640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932c0b5d40 0x7f9310004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.773+0000 7f9329e00640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93100039e0 0x7f9310003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.774+0000 7f932a800640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9310004d40 0x7f93100051c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9324009fc0 tx=0x7f9310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8990f7c272b27e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.774+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324009c10 con 0x7f9310004d40 2024-04-26T21:50:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.774+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324036070 con 0x7f9310004d40 2024-04-26T21:50:47.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.774+0000 7f9328a00640 1 -- 172.21.15.106:0/3536250364 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324031860 con 0x7f9310004d40 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 -- 172.21.15.106:0/3536250364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9310004d40 msgr2=0x7f93100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 --2- 172.21.15.106:0/3536250364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9310004d40 0x7f93100051c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9324009fc0 tx=0x7f9310005830 comp rx=0 tx=0).stop 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932a800640 1 -- 172.21.15.106:0/3536250364 reap_dead start 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 -- 172.21.15.106:0/3536250364 shutdown_connections 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 -- 172.21.15.106:0/3536250364 >> 172.21.15.106:0/3536250364 conn(0x7f932c06c740 msgr2=0x7f932c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 -- 172.21.15.106:0/3536250364 shutdown_connections 2024-04-26T21:50:47.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.775+0000 7f932b200640 1 -- 172.21.15.106:0/3536250364 wait complete. 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.777+0000 7f932b200640 1 Processor -- start 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.777+0000 7f932b200640 1 -- start start 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 0x7f932c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f932c158d20 0x7f932c1f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15f320 con 0x7f932c1467c0 2024-04-26T21:50:47.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15f1a0 con 0x7f932c158d20 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f932c06eb00 con 0x7f9310004d40 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48952/0 (socket says 172.21.15.106:48952) 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f932a800640 1 -- 172.21.15.106:0/1353134261 learned_addr learned my addr 172.21.15.106:0/1353134261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.778+0000 7f9329e00640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 0x7f932c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f9329400640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f932c158d20 0x7f932c1f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 -- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 msgr2=0x7f932c1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 0x7f932c1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 -- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f932c158d20 msgr2=0x7f932c1f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f932c158d20 0x7f932c1f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 -- 172.21.15.106:0/1353134261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c159230 con 0x7f932c1467c0 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f9329e00640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 0x7f932c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932a800640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f932400e230 tx=0x7f9324009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324009c10 con 0x7f932c1467c0 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324003bb0 con 0x7f932c1467c0 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324031920 con 0x7f932c1467c0 2024-04-26T21:50:47.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.779+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c159510 con 0x7f932c1467c0 2024-04-26T21:50:47.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.780+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1542c0 con 0x7f932c1467c0 2024-04-26T21:50:47.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.781+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c1544c0 con 0x7f932c1467c0 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.782+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f9324005d10 con 0x7f932c1467c0 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.783+0000 7f9322000640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 0x7f930c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.783+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f93240ac690 con 0x7f932c1467c0 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.783+0000 7f9329e00640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 0x7f930c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.784+0000 7f9329e00640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 0x7f930c06ef60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f931c007a30 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.787+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f932c1544c0 con 0x7f932c1467c0 2024-04-26T21:50:47.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.992+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f932c0afd40 con 0x7f930c06caa0 2024-04-26T21:50:47.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.993+0000 7f9322000640 1 -- 172.21.15.106:0/1353134261 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f932c0afd40 con 0x7f930c06caa0 2024-04-26T21:50:47.994 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T21:50:47.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 msgr2=0x7f930c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 0x7f930c06ef60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f931c007a30 tx=0x7f931c009040 comp rx=0 tx=0).stop 2024-04-26T21:50:47.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 msgr2=0x7f932c1f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f932400e230 tx=0x7f9324009750 comp rx=0 tx=0).stop 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 shutdown_connections 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f932c1467c0 0x7f932c1f5450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f932c158d20 0x7f932c1f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f930c06caa0 0x7f930c06ef60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 --2- 172.21.15.106:0/1353134261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9310004d40 0x7f932c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 >> 172.21.15.106:0/1353134261 conn(0x7f932c06c740 msgr2=0x7f932c161a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:47.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 shutdown_connections 2024-04-26T21:50:47.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:47.996+0000 7f932b200640 1 -- 172.21.15.106:0/1353134261 wait complete. 2024-04-26T21:50:48.015 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T21:50:48.016 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:48.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.223+0000 7ff7ab200640 1 Processor -- start 2024-04-26T21:50:48.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.223+0000 7ff7ab200640 1 -- start start 2024-04-26T21:50:48.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.224+0000 7ff7ab200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7ac0d2140 0x7ff7ac0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.224+0000 7ff7ab200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac0d3f00 0x7ff7ac0d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.224+0000 7ff7ab200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ac0d2e60 con 0x7ff7ac15f1c0 2024-04-26T21:50:48.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.224+0000 7ff7ab200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0d3a70 con 0x7ff7ac0d2140 2024-04-26T21:50:48.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.224+0000 7ff7ab200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d3140 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7ac0d2140 0x7ff7ac0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7ac0d2140 0x7ff7ac0d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53264/0 (socket says 172.21.15.106:53264) 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 -- 172.21.15.106:0/461483409 learned_addr learned my addr 172.21.15.106:0/461483409 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 -- 172.21.15.106:0/461483409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac0d3f00 msgr2=0x7ff7ac0d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 --2- 172.21.15.106:0/461483409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac0d3f00 0x7ff7ac0d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 -- 172.21.15.106:0/461483409 >> v1:172.21.15.204:6789/0 conn(0x7ff7ac15f1c0 legacy=0x7ff7ac0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.225+0000 7ff7a9400640 1 -- 172.21.15.106:0/461483409 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac0d3420 con 0x7ff7ac0d2140 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.226+0000 7ff7a9400640 1 --2- 172.21.15.106:0/461483409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac0d0760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff794009870 tx=0x7ff79402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e58600cbeb32e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7a8a00640 1 -- 172.21.15.106:0/461483409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794037070 con 0x7ff7ac0d2140 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7a8a00640 1 -- 172.21.15.106:0/461483409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff794032440 con 0x7ff7ac0d2140 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7a8a00640 1 -- 172.21.15.106:0/461483409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794032810 con 0x7ff7ac0d2140 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 -- 172.21.15.106:0/461483409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d2140 msgr2=0x7ff7ac0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 --2- 172.21.15.106:0/461483409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac0d0760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff794009870 tx=0x7ff79402a230 comp rx=0 tx=0).stop 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 -- 172.21.15.106:0/461483409 shutdown_connections 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 --2- 172.21.15.106:0/461483409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac0d3f00 0x7ff7ac0d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 --2- 172.21.15.106:0/461483409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac0d0760 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.227+0000 7ff7ab200640 1 -- 172.21.15.106:0/461483409 >> 172.21.15.106:0/461483409 conn(0x7ff7ac09e340 msgr2=0x7ff7ac0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.228+0000 7ff7ab200640 1 -- 172.21.15.106:0/461483409 shutdown_connections 2024-04-26T21:50:48.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.228+0000 7ff7ab200640 1 -- 172.21.15.106:0/461483409 wait complete. 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.229+0000 7ff7ab200640 1 Processor -- start 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.229+0000 7ff7ab200640 1 -- start start 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 0x7ff7ac15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac15f1c0 0x7ff7ac15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac15f720 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac15f5a0 con 0x7ff7ac0d2140 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.230+0000 7ff7ab200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0b48a0 con 0x7ff7ac15f1c0 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 0x7ff7ac15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 0x7ff7ac15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53270/0 (socket says 172.21.15.106:53270) 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 -- 172.21.15.106:0/4291759187 learned_addr learned my addr 172.21.15.106:0/4291759187 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9e00640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7aa800640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac15f1c0 0x7ff7ac15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 -- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac15f1c0 msgr2=0x7ff7ac15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7ac15f1c0 0x7ff7ac15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 -- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7ac0d2140 msgr2=0x7ff7ac160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac161070 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9e00640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7ac0d2140 0x7ff7ac160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:48.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.231+0000 7ff7a9400640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 0x7ff7ac15f8a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff794000c00 tx=0x7ff7940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.232+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794041070 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.232+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79403c870 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.232+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794037040 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.232+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac160ab0 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.232+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac1f8b60 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.233+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7ac0693c0 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.234+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff794044070 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a2000640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff78806caa0 0x7ff78806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ff7940ac7b0 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a9e00640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff78806caa0 0x7ff78806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a2000640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7ff788072960 0x7ff788074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff794034e90 con 0x7ff788072960 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.235+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff7940b3050 con 0x7ff7ac0d3f00 2024-04-26T21:50:48.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.236+0000 7ff7a9e00640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff78806caa0 0x7ff78806ef60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff79c007e90 tx=0x7ff79c0073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.236+0000 7ff7aa800640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7ff788072960 0x7ff788074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.237+0000 7ff7aa800640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7ff788072960 0x7ff788074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.239+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== osd.4 v2:172.21.15.204:6800/3289538728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff794034e90 con 0x7ff788072960 2024-04-26T21:50:48.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.253+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 --> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff7ac1f9120 con 0x7ff788072960 2024-04-26T21:50:48.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7a2000640 1 -- 172.21.15.106:0/4291759187 <== osd.4 v2:172.21.15.204:6800/3289538728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff7ac1f9120 con 0x7ff788072960 2024-04-26T21:50:48.256 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:48.256 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7ff788072960 msgr2=0x7ff788074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.204:6800/3289538728,v1:172.21.15.204:6801/3289538728] conn(0x7ff788072960 0x7ff788074d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff78806caa0 msgr2=0x7ff78806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff78806caa0 0x7ff78806ef60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff79c007e90 tx=0x7ff79c0073a0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 msgr2=0x7ff7ac15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7ab200640 1 --2- 172.21.15.106:0/4291759187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7ac0d3f00 0x7ff7ac15f8a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff794000c00 tx=0x7ff7940046c0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.255+0000 7ff7aa800640 1 -- 172.21.15.106:0/4291759187 reap_dead start 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.256+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 shutdown_connections 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.256+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 >> 172.21.15.106:0/4291759187 conn(0x7ff7ac09e340 msgr2=0x7ff7ac0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.256+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 shutdown_connections 2024-04-26T21:50:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.256+0000 7ff7ab200640 1 -- 172.21.15.106:0/4291759187 wait complete. 2024-04-26T21:50:48.274 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T21:50:48.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.475+0000 7f7d80200640 1 Processor -- start 2024-04-26T21:50:48.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.475+0000 7f7d80200640 1 -- start start 2024-04-26T21:50:48.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.477+0000 7f7d80200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d781598a0 0x7f7d781638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.477+0000 7f7d80200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d780b5fb0 con 0x7f7d7815fe50 2024-04-26T21:50:48.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.477+0000 7f7d80200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d78160370 con 0x7f7d7806c770 2024-04-26T21:50:48.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.477+0000 7f7d80200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d78159c80 con 0x7f7d781598a0 2024-04-26T21:50:48.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d7e200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7d7815fe50 0x7f7d780add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54914/0 (socket says 172.21.15.106:54914) 2024-04-26T21:50:48.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d7e200640 1 -- 172.21.15.106:0/1105717652 learned_addr learned my addr 172.21.15.106:0/1105717652 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:48.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d7d800640 1 --2- 172.21.15.106:0/1105717652 >> v2:172.21.15.204:3300/0 conn(0x7f7d781598a0 0x7f7d781638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 570024544 0 0) 0x7f7d780b5fb0 con 0x7f7d7815fe50 2024-04-26T21:50:48.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d60003520 con 0x7f7d7815fe50 2024-04-26T21:50:48.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.478+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051467084 0 0) 0x7f7d60003520 con 0x7f7d7815fe50 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 >> v2:172.21.15.204:3300/0 conn(0x7f7d781598a0 msgr2=0x7f7d781638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 --2- 172.21.15.106:0/1105717652 >> v2:172.21.15.204:3300/0 conn(0x7f7d781598a0 0x7f7d781638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 >> v1:172.21.15.204:6789/0 conn(0x7f7d7806c770 legacy=0x7f7d7815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d78162dd0 con 0x7f7d7815fe50 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f7d700019f0 con 0x7f7d7815fe50 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 >> v1:172.21.15.106:6789/0 conn(0x7f7d7815fe50 legacy=0x7f7d780add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d60003a00 0x7f7d60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d60005210 0x7f7d6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78159c80 con 0x7f7d60004800 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6000c160 con 0x7f7d60003a00 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6000c360 con 0x7f7d60005210 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7d800640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d60003a00 0x7f7d60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 -- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d60005210 msgr2=0x7f7d6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d60005210 0x7f7d6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 -- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d60003a00 msgr2=0x7f7d60003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d60003a00 0x7f7d60003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.479+0000 7f7d7ce00640 1 -- 172.21.15.106:0/1105717652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d60003520 con 0x7f7d60004800 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.480+0000 7f7d7e200640 1 -- 172.21.15.106:0/1105717652 reap_dead start 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.480+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d60004c80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7d64001720 tx=0x7f7d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282677c84a6aa44b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64001370 con 0x7f7d60004800 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d640052d0 con 0x7f7d60004800 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d77e00640 1 -- 172.21.15.106:0/1105717652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402ab90 con 0x7f7d60004800 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 -- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 msgr2=0x7f7d60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d60004c80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7d64001720 tx=0x7f7d60005830 comp rx=0 tx=0).stop 2024-04-26T21:50:48.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 -- 172.21.15.106:0/1105717652 shutdown_connections 2024-04-26T21:50:48.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 --2- 172.21.15.106:0/1105717652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d60004c80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 -- 172.21.15.106:0/1105717652 >> 172.21.15.106:0/1105717652 conn(0x7f7d780b2d50 msgr2=0x7f7d780b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 -- 172.21.15.106:0/1105717652 shutdown_connections 2024-04-26T21:50:48.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.481+0000 7f7d80200640 1 -- 172.21.15.106:0/1105717652 wait complete. 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.483+0000 7f7d80200640 1 Processor -- start 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.483+0000 7f7d80200640 1 -- start start 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d780cfa60 0x7f7d780af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d780cfec0 0x7f7d780af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78160650 con 0x7f7d60004800 2024-04-26T21:50:48.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78163390 con 0x7f7d780cfec0 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.484+0000 7f7d80200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d781638b0 con 0x7f7d780cfa60 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53290/0 (socket says 172.21.15.106:53290) 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 -- 172.21.15.106:0/4167385946 learned_addr learned my addr 172.21.15.106:0/4167385946 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 -- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d780cfa60 msgr2=0x7f7d780af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d780cfa60 0x7f7d780af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 -- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d780cfec0 msgr2=0x7f7d780af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d780cfec0 0x7f7d780af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 -- 172.21.15.106:0/4167385946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7809cb90 con 0x7f7d60004800 2024-04-26T21:50:48.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.485+0000 7f7d7d800640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0108e0 tx=0x7f7d6c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.486+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00f040 con 0x7f7d60004800 2024-04-26T21:50:48.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.486+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00eb60 con 0x7f7d60004800 2024-04-26T21:50:48.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.486+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c01c400 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.486+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7809ce70 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.486+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7809d430 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.487+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d78069540 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.488+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f7d6c01c5a0 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.489+0000 7f7d75600640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 0x7f7d5806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.489+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f7d6c08c420 con 0x7f7d60004800 2024-04-26T21:50:48.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.489+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 0x7f7d5806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.490+0000 7f7d7ce00640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 0x7f7d5806ef60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d64032000 tx=0x7f7d6402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.494+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d6c018030 con 0x7f7d60004800 2024-04-26T21:50:48.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.691+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f7d781f9630 con 0x7f7d5806caa0 2024-04-26T21:50:48.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.692+0000 7f7d75600640 1 -- 172.21.15.106:0/4167385946 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7d64005150 con 0x7f7d5806caa0 2024-04-26T21:50:48.693 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T21:50:48.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 msgr2=0x7f7d5806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 0x7f7d5806ef60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d64032000 tx=0x7f7d6402c000 comp rx=0 tx=0).stop 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 msgr2=0x7f7d7809d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0108e0 tx=0x7f7d6c010da0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 shutdown_connections 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d780cfec0 0x7f7d780af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f7d5806caa0 0x7f7d5806ef60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d780cfa60 0x7f7d780af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 --2- 172.21.15.106:0/4167385946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d60004800 0x7f7d7809d950 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.694+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 >> 172.21.15.106:0/4167385946 conn(0x7f7d780b2d50 msgr2=0x7f7d780d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.695+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 shutdown_connections 2024-04-26T21:50:48.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.695+0000 7f7d80200640 1 -- 172.21.15.106:0/4167385946 wait complete. 2024-04-26T21:50:48.715 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T21:50:48.715 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:48.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.920+0000 7fa162c00640 1 Processor -- start 2024-04-26T21:50:48.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.920+0000 7fa162c00640 1 -- start start 2024-04-26T21:50:48.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.921+0000 7fa162c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 0x7fa15c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.921+0000 7fa162c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c06df60 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.921+0000 7fa162c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c146ba0 con 0x7fa15c143470 2024-04-26T21:50:48.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.921+0000 7fa162c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c06e200 con 0x7fa15c143850 2024-04-26T21:50:48.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa162200640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa15c0cb8b0 0x7fa15c0723c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54930/0 (socket says 172.21.15.106:54930) 2024-04-26T21:50:48.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa162200640 1 -- 172.21.15.106:0/3766198215 learned_addr learned my addr 172.21.15.106:0/3766198215 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:48.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa160e00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 0x7fa15c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695466448 0 0) 0x7fa15c06df60 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa140003520 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450372165 0 0) 0x7fa140003520 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 msgr2=0x7fa15c06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 0x7fa15c06fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 >> v1:172.21.15.204:6789/0 conn(0x7fa15c143470 legacy=0x7fa15c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c06e4e0 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7fa154001690 con 0x7fa15c0cb8b0 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.922+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 >> v1:172.21.15.106:6789/0 conn(0x7fa15c0cb8b0 legacy=0x7fa15c0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1400039e0 0x7fa140003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa140004800 0x7fa140004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 0x7fa140005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c06e200 con 0x7fa1400039e0 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa14000baf0 con 0x7fa140004800 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa14000bcf0 con 0x7fa15c143850 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1400039e0 0x7fa140003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 -- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 msgr2=0x7fa140005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa15c143850 0x7fa140005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 -- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa140004800 msgr2=0x7fa140004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa160e00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa140004800 0x7fa140004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa140004800 0x7fa140004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.923+0000 7fa161800640 1 -- 172.21.15.106:0/3766198215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa140003520 con 0x7fa1400039e0 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.924+0000 7fa161800640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1400039e0 0x7fa140003e00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001e50 tx=0x7fa140005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb96c1943ee85d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.924+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c001a70 con 0x7fa1400039e0 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14c003970 con 0x7fa1400039e0 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa15be00640 1 -- 172.21.15.106:0/3766198215 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c003d60 con 0x7fa1400039e0 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 -- 172.21.15.106:0/3766198215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1400039e0 msgr2=0x7fa140003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 --2- 172.21.15.106:0/3766198215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1400039e0 0x7fa140003e00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001e50 tx=0x7fa140005830 comp rx=0 tx=0).stop 2024-04-26T21:50:48.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162200640 1 -- 172.21.15.106:0/3766198215 reap_dead start 2024-04-26T21:50:48.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 -- 172.21.15.106:0/3766198215 shutdown_connections 2024-04-26T21:50:48.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 -- 172.21.15.106:0/3766198215 >> 172.21.15.106:0/3766198215 conn(0x7fa15c0d0b50 msgr2=0x7fa15c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 -- 172.21.15.106:0/3766198215 shutdown_connections 2024-04-26T21:50:48.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.925+0000 7fa162c00640 1 -- 172.21.15.106:0/3766198215 wait complete. 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.927+0000 7fa162c00640 1 Processor -- start 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.927+0000 7fa162c00640 1 -- start start 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400039e0 0x7fa15c13f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c0b4200 0x7fa15c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 0x7fa15c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0cbd00 con 0x7fa15c145530 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c070180 con 0x7fa15c0b4200 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0cbe80 con 0x7fa1400039e0 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 0x7fa15c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 0x7fa15c1459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53300/0 (socket says 172.21.15.106:53300) 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.928+0000 7fa162200640 1 -- 172.21.15.106:0/3067933923 learned_addr learned my addr 172.21.15.106:0/3067933923 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa160e00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c0b4200 0x7fa15c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa161800640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400039e0 0x7fa15c13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 -- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400039e0 msgr2=0x7fa15c13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400039e0 0x7fa15c13f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 -- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c0b4200 msgr2=0x7fa15c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c0b4200 0x7fa15c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c0b4d70 con 0x7fa15c145530 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa160e00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c0b4200 0x7fa15c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162200640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 0x7fa15c1459b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa140004780 tx=0x7fa15400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154007440 con 0x7fa15c145530 2024-04-26T21:50:48.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1540075e0 con 0x7fa15c145530 2024-04-26T21:50:48.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154004a00 con 0x7fa15c145530 2024-04-26T21:50:48.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.929+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c0b5050 con 0x7fa15c145530 2024-04-26T21:50:48.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.930+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c0aa760 con 0x7fa15c145530 2024-04-26T21:50:48.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.930+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa15c145ef0 con 0x7fa15c145530 2024-04-26T21:50:48.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.932+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa154004ba0 con 0x7fa15c145530 2024-04-26T21:50:48.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.932+0000 7fa159600640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa13c06caa0 0x7fa13c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.933+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa15408b0d0 con 0x7fa15c145530 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.933+0000 7fa161800640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa13c06caa0 0x7fa13c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.933+0000 7fa159600640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa13c072960 0x7fa13c074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.933+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa13c075430 con 0x7fa13c072960 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.933+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa154092050 con 0x7fa15c145530 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.934+0000 7fa160e00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa13c072960 0x7fa13c074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.934+0000 7fa161800640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa13c06caa0 0x7fa13c06ef60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa14c032000 tx=0x7fa14c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.935+0000 7fa160e00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa13c072960 0x7fa13c074d80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.937+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== osd.5 v2:172.21.15.204:6816/885497049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa13c075430 con 0x7fa13c072960 2024-04-26T21:50:48.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.951+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 --> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa15c06d1f0 con 0x7fa13c072960 2024-04-26T21:50:48.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.952+0000 7fa159600640 1 -- 172.21.15.106:0/3067933923 <== osd.5 v2:172.21.15.204:6816/885497049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa15c06d1f0 con 0x7fa13c072960 2024-04-26T21:50:48.953 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:48.953 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:48.953 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:48.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.952+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa13c072960 msgr2=0x7fa13c074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.952+0000 7fa162c00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.204:6816/885497049,v1:172.21.15.204:6817/885497049] conn(0x7fa13c072960 0x7fa13c074d80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa13c06caa0 msgr2=0x7fa13c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fa13c06caa0 0x7fa13c06ef60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa14c032000 tx=0x7fa14c02e000 comp rx=0 tx=0).stop 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 msgr2=0x7fa15c1459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 --2- 172.21.15.106:0/3067933923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa15c145530 0x7fa15c1459b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa140004780 tx=0x7fa15400c960 comp rx=0 tx=0).stop 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162200640 1 -- 172.21.15.106:0/3067933923 reap_dead start 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 shutdown_connections 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 >> 172.21.15.106:0/3067933923 conn(0x7fa15c0d0b50 msgr2=0x7fa15c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 shutdown_connections 2024-04-26T21:50:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:48.953+0000 7fa162c00640 1 -- 172.21.15.106:0/3067933923 wait complete. 2024-04-26T21:50:48.971 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T21:50:49.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.163+0000 7ff6bec00640 1 Processor -- start 2024-04-26T21:50:49.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.164+0000 7ff6bec00640 1 -- start start 2024-04-26T21:50:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.165+0000 7ff6bec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff6b806ce80 0x7ff6b81638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.165+0000 7ff6bec00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b80b5fb0 con 0x7ff6b81598a0 2024-04-26T21:50:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.165+0000 7ff6bec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b8160370 con 0x7ff6b815f740 2024-04-26T21:50:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.165+0000 7ff6bec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b8159c80 con 0x7ff6b806ce80 2024-04-26T21:50:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff6b81598a0 0x7ff6b806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54936/0 (socket says 172.21.15.106:54936) 2024-04-26T21:50:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7e00640 1 -- 172.21.15.106:0/3669574458 learned_addr learned my addr 172.21.15.106:0/3669574458 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:49.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672663861 0 0) 0x7ff6b80b5fb0 con 0x7ff6b81598a0 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6af400640 1 --2- 172.21.15.106:0/3669574458 >> v2:172.21.15.204:3300/0 conn(0x7ff6b806ce80 0x7ff6b81638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff69c003520 con 0x7ff6b81598a0 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 680478201 0 0) 0x7ff69c003520 con 0x7ff6b81598a0 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 >> v2:172.21.15.204:3300/0 conn(0x7ff6b806ce80 msgr2=0x7ff6b81638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 --2- 172.21.15.106:0/3669574458 >> v2:172.21.15.204:3300/0 conn(0x7ff6b806ce80 0x7ff6b81638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 >> v1:172.21.15.204:6789/0 conn(0x7ff6b815f740 legacy=0x7ff6b815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b8162dd0 con 0x7ff6b81598a0 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.166+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7ff6a8001670 con 0x7ff6b81598a0 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 >> v1:172.21.15.106:6789/0 conn(0x7ff6b81598a0 legacy=0x7ff6b806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 0x7ff69c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff69c004800 0x7ff69c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0052e0 0x7ff69c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8159c80 con 0x7ff69c0039b0 2024-04-26T21:50:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c00c190 con 0x7ff69c0052e0 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c00c390 con 0x7ff69c004800 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 0x7ff69c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6af400640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff69c004800 0x7ff69c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 -- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff69c004800 msgr2=0x7ff69c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff69c004800 0x7ff69c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 -- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0052e0 msgr2=0x7ff69c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0052e0 0x7ff69c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6b7e00640 1 -- 172.21.15.106:0/3669574458 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c003520 con 0x7ff69c0039b0 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.167+0000 7ff6bcc00640 1 -- 172.21.15.106:0/3669574458 reap_dead start 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.168+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 0x7ff69c003db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c210 tx=0x7ff69c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdb6b737d6b9967 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8007d10 con 0x7ff69c0039b0 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a80058d0 con 0x7ff69c0039b0 2024-04-26T21:50:49.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6b7400640 1 -- 172.21.15.106:0/3669574458 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8005d20 con 0x7ff69c0039b0 2024-04-26T21:50:49.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 -- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 msgr2=0x7ff69c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 0x7ff69c003db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c210 tx=0x7ff69c005830 comp rx=0 tx=0).stop 2024-04-26T21:50:49.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 -- 172.21.15.106:0/3669574458 shutdown_connections 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 --2- 172.21.15.106:0/3669574458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff69c0039b0 0x7ff69c003db0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 -- 172.21.15.106:0/3669574458 >> 172.21.15.106:0/3669574458 conn(0x7ff6b80b2d50 msgr2=0x7ff6b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.169+0000 7ff6bec00640 1 -- 172.21.15.106:0/3669574458 shutdown_connections 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.170+0000 7ff6bec00640 1 -- 172.21.15.106:0/3669574458 wait complete. 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.171+0000 7ff6bec00640 1 Processor -- start 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.171+0000 7ff6bec00640 1 -- start start 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0039b0 0x7ff6b81e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6b809e110 0x7ff6b80cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8160650 con 0x7ff6b8135070 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8163390 con 0x7ff69c0039b0 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.172+0000 7ff6bec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b81638b0 con 0x7ff6b809e110 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53326/0 (socket says 172.21.15.106:53326) 2024-04-26T21:50:49.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 -- 172.21.15.106:0/2838274587 learned_addr learned my addr 172.21.15.106:0/2838274587 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0039b0 0x7ff6b81e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 -- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6b809e110 msgr2=0x7ff6b80cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6b809e110 0x7ff6b80cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 -- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0039b0 msgr2=0x7ff6b81e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0039b0 0x7ff6b81e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 -- 172.21.15.106:0/2838274587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b81489f0 con 0x7ff6b8135070 2024-04-26T21:50:49.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.173+0000 7ff6bcc00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8162c70 tx=0x7ff6b000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.174+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0017070 con 0x7ff6b8135070 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.174+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0007070 con 0x7ff6b8135070 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.174+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0007460 con 0x7ff6b8135070 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.174+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b81e9170 con 0x7ff6b8135070 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.174+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b809d910 con 0x7ff6b8135070 2024-04-26T21:50:49.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.175+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b81591a0 con 0x7ff6b8135070 2024-04-26T21:50:49.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.176+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff6b001a070 con 0x7ff6b8135070 2024-04-26T21:50:49.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.177+0000 7ff6b4c00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 0x7ff69406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.177+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ff6b008adc0 con 0x7ff6b8135070 2024-04-26T21:50:49.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.177+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 0x7ff69406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.178+0000 7ff6b7e00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 0x7ff69406ef60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8033000 tx=0x7ff6a8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.181+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6b81489f0 con 0x7ff6b8135070 2024-04-26T21:50:49.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.380+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ff6b8069540 con 0x7ff69406caa0 2024-04-26T21:50:49.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.381+0000 7ff6b4c00640 1 -- 172.21.15.106:0/2838274587 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff6b8069540 con 0x7ff69406caa0 2024-04-26T21:50:49.382 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 msgr2=0x7ff69406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 0x7ff69406ef60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8033000 tx=0x7ff6a8009030 comp rx=0 tx=0).stop 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 msgr2=0x7ff6b81354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8162c70 tx=0x7ff6b000c960 comp rx=0 tx=0).stop 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 shutdown_connections 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6b8135070 0x7ff6b81354f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff69406caa0 0x7ff69406ef60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6b809e110 0x7ff6b80cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 --2- 172.21.15.106:0/2838274587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff69c0039b0 0x7ff6b81e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.383+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 >> 172.21.15.106:0/2838274587 conn(0x7ff6b80b2d50 msgr2=0x7ff6b8068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.384+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 shutdown_connections 2024-04-26T21:50:49.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.384+0000 7ff6bec00640 1 -- 172.21.15.106:0/2838274587 wait complete. 2024-04-26T21:50:49.403 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T21:50:49.403 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.609+0000 7f1f90c00640 1 Processor -- start 2024-04-26T21:50:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.609+0000 7f1f90c00640 1 -- start start 2024-04-26T21:50:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f8c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f90c00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0ae160 con 0x7f1f8c0aff10 2024-04-26T21:50:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f90c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c06ff80 con 0x7f1f8c06ef60 2024-04-26T21:50:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f90c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0d0520 con 0x7f1f8c15d170 2024-04-26T21:50:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f8a800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1f8c0aff10 0x7f1f8c072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54950/0 (socket says 172.21.15.106:54950) 2024-04-26T21:50:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.611+0000 7f1f8a800640 1 -- 172.21.15.106:0/3126952280 learned_addr learned my addr 172.21.15.106:0/3126952280 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f8c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210717079 0 0) 0x7f1f8c0ae160 con 0x7f1f8c0aff10 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f70003520 con 0x7f1f8c0aff10 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3600700232 0 0) 0x7f1f70003520 con 0x7f1f8c0aff10 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 msgr2=0x7f1f8c0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f8c0cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 >> v1:172.21.15.204:6789/0 conn(0x7f1f8c06ef60 legacy=0x7f1f8c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c070260 con 0x7f1f8c0aff10 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f1f840019f0 con 0x7f1f8c0aff10 2024-04-26T21:50:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 >> v1:172.21.15.106:6789/0 conn(0x7f1f8c0aff10 legacy=0x7f1f8c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f70003f60 0x7f1f70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.612+0000 7f1f89400640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70004d40 0x7f1f700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c0d0520 con 0x7f1f70003f60 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7000bb50 con 0x7f1f70004d40 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7000bd50 con 0x7f1f8c15d170 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f70003f60 0x7f1f70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f89e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f8a800640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70004d40 0x7f1f700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 -- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 msgr2=0x7f1f700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c15d170 0x7f1f700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 -- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70004d40 msgr2=0x7f1f700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70004d40 0x7f1f700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.613+0000 7f1f83e00640 1 -- 172.21.15.106:0/3126952280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70003520 con 0x7f1f70003f60 2024-04-26T21:50:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.614+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f70003f60 0x7f1f70004800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f74001750 tx=0x7f1f70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93efc1eef38dbf60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74001370 con 0x7f1f70003f60 2024-04-26T21:50:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f740052d0 con 0x7f1f70003f60 2024-04-26T21:50:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f89400640 1 -- 172.21.15.106:0/3126952280 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7402abe0 con 0x7f1f70003f60 2024-04-26T21:50:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f90c00640 1 -- 172.21.15.106:0/3126952280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f70003f60 msgr2=0x7f1f70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f90c00640 1 --2- 172.21.15.106:0/3126952280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f70003f60 0x7f1f70004800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f74001750 tx=0x7f1f70005830 comp rx=0 tx=0).stop 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f8a800640 1 -- 172.21.15.106:0/3126952280 reap_dead start 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f90c00640 1 -- 172.21.15.106:0/3126952280 shutdown_connections 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f90c00640 1 -- 172.21.15.106:0/3126952280 >> 172.21.15.106:0/3126952280 conn(0x7f1f8c0d0c40 msgr2=0x7f1f8c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.615+0000 7f1f90c00640 1 -- 172.21.15.106:0/3126952280 shutdown_connections 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.616+0000 7f1f90c00640 1 -- 172.21.15.106:0/3126952280 wait complete. 2024-04-26T21:50:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.617+0000 7f1f90c00640 1 Processor -- start 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.617+0000 7f1f90c00640 1 -- start start 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70003f60 0x7f1f8c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c1434e0 0x7f1f8c15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 0x7f1f8c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06fd30 con 0x7f1f8c13fd60 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c15d550 con 0x7f1f70003f60 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.618+0000 7f1f90c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0add80 con 0x7f1f8c1434e0 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 0x7f1f8c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 0x7f1f8c1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53348/0 (socket says 172.21.15.106:53348) 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 -- 172.21.15.106:0/3445025267 learned_addr learned my addr 172.21.15.106:0/3445025267 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c1434e0 0x7f1f8c15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f89e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70003f60 0x7f1f8c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 -- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c1434e0 msgr2=0x7f1f8c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f8c1434e0 0x7f1f8c15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 -- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70003f60 msgr2=0x7f1f8c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70003f60 0x7f1f8c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c143940 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.619+0000 7f1f8a800640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 0x7f1f8c1401e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000f80 tx=0x7f1f8400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.620+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400ee20 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.620+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f84018070 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.620+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c0ce760 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.620+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84013660 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.620+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c0ced20 con 0x7f1f8c13fd60 2024-04-26T21:50:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.621+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f8c069650 con 0x7f1f8c13fd60 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.622+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f1f84013800 con 0x7f1f8c13fd60 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f82000640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f6806caa0 0x7f1f6806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f1f8408bef0 con 0x7f1f8c13fd60 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f89e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f6806caa0 0x7f1f6806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f82000640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f1f68072960 0x7f1f68074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f68075430 con 0x7f1f68072960 2024-04-26T21:50:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.623+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1f8408e510 con 0x7f1f8c13fd60 2024-04-26T21:50:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.624+0000 7f1f89e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f6806caa0 0x7f1f6806ef60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009950 tx=0x7f1f7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.624+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f1f68072960 0x7f1f68074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.625+0000 7f1f83e00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f1f68072960 0x7f1f68074d80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.627+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== osd.6 v2:172.21.15.204:6821/3330975067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f68075430 con 0x7f1f68072960 2024-04-26T21:50:49.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.641+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 --> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1f8c0b0550 con 0x7f1f68072960 2024-04-26T21:50:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.642+0000 7f1f82000640 1 -- 172.21.15.106:0/3445025267 <== osd.6 v2:172.21.15.204:6821/3330975067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1f8c0b0550 con 0x7f1f68072960 2024-04-26T21:50:49.644 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:49.644 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:49.644 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f1f68072960 msgr2=0x7f1f68074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.204:6821/3330975067,v1:172.21.15.204:6823/3330975067] conn(0x7f1f68072960 0x7f1f68074d80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f6806caa0 msgr2=0x7f1f6806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f1f6806caa0 0x7f1f6806ef60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009950 tx=0x7f1f7c008040 comp rx=0 tx=0).stop 2024-04-26T21:50:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 msgr2=0x7f1f8c1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 --2- 172.21.15.106:0/3445025267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f8c13fd60 0x7f1f8c1401e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000f80 tx=0x7f1f8400e530 comp rx=0 tx=0).stop 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f8a800640 1 -- 172.21.15.106:0/3445025267 reap_dead start 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 shutdown_connections 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 >> 172.21.15.106:0/3445025267 conn(0x7f1f8c0d0c40 msgr2=0x7f1f8c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 shutdown_connections 2024-04-26T21:50:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.643+0000 7f1f90c00640 1 -- 172.21.15.106:0/3445025267 wait complete. 2024-04-26T21:50:49.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T21:50:49.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.859+0000 7ff53f600640 1 Processor -- start 2024-04-26T21:50:49.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.859+0000 7ff53f600640 1 -- start start 2024-04-26T21:50:49.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.860+0000 7ff53f600640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff53806ce80 0x7ff5380add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.860+0000 7ff53f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5380cd650 0x7ff5380b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.860+0000 7ff53f600640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5380af9a0 con 0x7ff53815cdf0 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.860+0000 7ff53f600640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff5380ae2c0 con 0x7ff53806ce80 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.860+0000 7ff53f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff5380b05b0 con 0x7ff5380cd650 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff53806ce80 0x7ff5380add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff53806ce80 0x7ff5380add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53354/0 (socket says 172.21.15.106:53354) 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 -- 172.21.15.106:0/691825057 learned_addr learned my addr 172.21.15.106:0/691825057 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff53cc00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff53815cdf0 0x7ff53806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54952/0 (socket says 172.21.15.106:54952) 2024-04-26T21:50:49.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff53d600640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5380cd650 0x7ff5380b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 -- 172.21.15.106:0/691825057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5380cd650 msgr2=0x7ff5380b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:49.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5380cd650 0x7ff5380b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 -- 172.21.15.106:0/691825057 >> v1:172.21.15.106:6789/0 conn(0x7ff53815cdf0 legacy=0x7ff53806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.861+0000 7ff537e00640 1 -- 172.21.15.106:0/691825057 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5380afc80 con 0x7ff53806ce80 2024-04-26T21:50:49.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.863+0000 7ff537e00640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff53806ce80 0x7ff5380add80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e50 tx=0x7ff52402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee06b0f9ba829ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.863+0000 7ff537400640 1 -- 172.21.15.106:0/691825057 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524036070 con 0x7ff53806ce80 2024-04-26T21:50:49.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.863+0000 7ff537400640 1 -- 172.21.15.106:0/691825057 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff524031600 con 0x7ff53806ce80 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.863+0000 7ff537400640 1 -- 172.21.15.106:0/691825057 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240319d0 con 0x7ff53806ce80 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 -- 172.21.15.106:0/691825057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff53806ce80 msgr2=0x7ff5380add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff53806ce80 0x7ff5380add80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e50 tx=0x7ff52402a730 comp rx=0 tx=0).stop 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 -- 172.21.15.106:0/691825057 shutdown_connections 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5380cd650 0x7ff5380b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 --2- 172.21.15.106:0/691825057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff53806ce80 0x7ff5380add80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 -- 172.21.15.106:0/691825057 >> 172.21.15.106:0/691825057 conn(0x7ff5380b2d50 msgr2=0x7ff5380b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 -- 172.21.15.106:0/691825057 shutdown_connections 2024-04-26T21:50:49.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.864+0000 7ff53f600640 1 -- 172.21.15.106:0/691825057 wait complete. 2024-04-26T21:50:49.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.865+0000 7ff53f600640 1 Processor -- start 2024-04-26T21:50:49.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.866+0000 7ff53f600640 1 -- start start 2024-04-26T21:50:49.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53806ce80 0x7ff53809c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 0x7ff5381f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff53815d1d0 con 0x7ff5380cd650 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5380b0240 con 0x7ff53806ce80 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff53815d350 con 0x7ff53815cdf0 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53362/0 (socket says 172.21.15.106:53362) 2024-04-26T21:50:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 -- 172.21.15.106:0/1113701602 learned_addr learned my addr 172.21.15.106:0/1113701602 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53d600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 0x7ff5381f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53cc00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53806ce80 0x7ff53809c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 -- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 msgr2=0x7ff5381f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 0x7ff5381f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 -- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53806ce80 msgr2=0x7ff53809c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53806ce80 0x7ff53809c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff537e00640 1 -- 172.21.15.106:0/1113701602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53809cd50 con 0x7ff5380cd650 2024-04-26T21:50:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.867+0000 7ff53d600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 0x7ff5381f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff537e00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff524034040 tx=0x7ff524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524036050 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff524042070 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53809d940 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524009510 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.868+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53809bd30 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.870+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff538069540 con 0x7ff5380cd650 2024-04-26T21:50:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.871+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff524007c70 con 0x7ff5380cd650 2024-04-26T21:50:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.872+0000 7ff534c00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 0x7ff51806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.872+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ff5240acb10 con 0x7ff5380cd650 2024-04-26T21:50:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.872+0000 7ff53cc00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 0x7ff51806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.873+0000 7ff53cc00640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 0x7ff51806ef60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff528000f60 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:49.877+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff524045360 con 0x7ff5380cd650 2024-04-26T21:50:50.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.075+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff5380b5030 con 0x7ff51806caa0 2024-04-26T21:50:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.076+0000 7ff534c00640 1 -- 172.21.15.106:0/1113701602 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff5380b5030 con 0x7ff51806caa0 2024-04-26T21:50:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.079+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 msgr2=0x7ff51806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.079+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 0x7ff51806ef60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff528000f60 tx=0x7ff528009040 comp rx=0 tx=0).stop 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 msgr2=0x7ff53809cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff524034040 tx=0x7ff524004a40 comp rx=0 tx=0).stop 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 shutdown_connections 2024-04-26T21:50:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ff51806caa0 0x7ff51806ef60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff53815cdf0 0x7ff5381f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5380cd650 0x7ff53809cf60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 --2- 172.21.15.106:0/1113701602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53806ce80 0x7ff53809c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 >> 172.21.15.106:0/1113701602 conn(0x7ff5380b2d50 msgr2=0x7ff53806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 shutdown_connections 2024-04-26T21:50:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.080+0000 7ff53f600640 1 -- 172.21.15.106:0/1113701602 wait complete. 2024-04-26T21:50:50.099 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T21:50:50.099 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:50:50.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.297+0000 7fc61be00640 1 Processor -- start 2024-04-26T21:50:50.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.297+0000 7fc61be00640 1 -- start start 2024-04-26T21:50:50.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc61c143850 0x7fc61c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61be00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c06df60 con 0x7fc61c0cb8b0 2024-04-26T21:50:50.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c146ba0 con 0x7fc61c143470 2024-04-26T21:50:50.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc61c06e200 con 0x7fc61c143850 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61b400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc61c0cb8b0 0x7fc61c0723c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54966/0 (socket says 172.21.15.106:54966) 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.299+0000 7fc61b400640 1 -- 172.21.15.106:0/3924507704 learned_addr learned my addr 172.21.15.106:0/3924507704 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc61a000640 1 --2- 172.21.15.106:0/3924507704 >> v2:172.21.15.204:3300/0 conn(0x7fc61c143850 0x7fc61c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc619600640 1 -- 172.21.15.106:0/3924507704 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320004456 0 0) 0x7fc61c06df60 con 0x7fc61c0cb8b0 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc619600640 1 -- 172.21.15.106:0/3924507704 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc600003520 con 0x7fc61c0cb8b0 2024-04-26T21:50:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc61a000640 1 -- 172.21.15.106:0/3924507704 >> v1:172.21.15.204:6789/0 conn(0x7fc61c143470 legacy=0x7fc61c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc61a000640 1 -- 172.21.15.106:0/3924507704 >> v1:172.21.15.106:6789/0 conn(0x7fc61c0cb8b0 legacy=0x7fc61c0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.300+0000 7fc61a000640 1 -- 172.21.15.106:0/3924507704 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c06e4e0 con 0x7fc61c143850 2024-04-26T21:50:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.302+0000 7fc61a000640 1 --2- 172.21.15.106:0/3924507704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c06fb10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6080097a0 tx=0x7fc60802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07838fd07ecb256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.302+0000 7fc619600640 1 -- 172.21.15.106:0/3924507704 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608037070 con 0x7fc61c143850 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc619600640 1 -- 172.21.15.106:0/3924507704 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc608032440 con 0x7fc61c143850 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc619600640 1 -- 172.21.15.106:0/3924507704 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608032810 con 0x7fc61c143850 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 -- 172.21.15.106:0/3924507704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 msgr2=0x7fc61c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 --2- 172.21.15.106:0/3924507704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c06fb10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6080097a0 tx=0x7fc60802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 -- 172.21.15.106:0/3924507704 shutdown_connections 2024-04-26T21:50:50.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 --2- 172.21.15.106:0/3924507704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c06fb10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 -- 172.21.15.106:0/3924507704 >> 172.21.15.106:0/3924507704 conn(0x7fc61c0d0b50 msgr2=0x7fc61c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 -- 172.21.15.106:0/3924507704 shutdown_connections 2024-04-26T21:50:50.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.303+0000 7fc61be00640 1 -- 172.21.15.106:0/3924507704 wait complete. 2024-04-26T21:50:50.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.305+0000 7fc61be00640 1 Processor -- start 2024-04-26T21:50:50.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.306+0000 7fc61be00640 1 -- start start 2024-04-26T21:50:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.306+0000 7fc61be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc61c0cb8b0 0x7fc61c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 0x7fc61c1e3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0cbd00 con 0x7fc61c143470 2024-04-26T21:50:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c070180 con 0x7fc61c143850 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c0cbe80 con 0x7fc61c0cb8b0 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 0x7fc61c1e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 0x7fc61c1e3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53370/0 (socket says 172.21.15.106:53370) 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 -- 172.21.15.106:0/4017929297 learned_addr learned my addr 172.21.15.106:0/4017929297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 -- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc61c0cb8b0 msgr2=0x7fc61c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc61c0cb8b0 0x7fc61c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61b400640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 -- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 msgr2=0x7fc61c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61a000640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c1e24c0 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.307+0000 7fc61b400640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc61c143850 0x7fc61c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc61a000640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 0x7fc61c1e3560 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098c0 tx=0x7fc6080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608037040 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc608030500 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c1e27a0 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080308d0 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.308+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c1e2d60 con 0x7fc61c143470 2024-04-26T21:50:50.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.309+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc61c06d1f0 con 0x7fc61c143470 2024-04-26T21:50:50.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.311+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc608044020 con 0x7fc61c143470 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc612a00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc5fc06caa0 0x7fc5fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc6080ac970 con 0x7fc61c143470 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc61aa00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc5fc06caa0 0x7fc5fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc612a00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fc5fc072960 0x7fc5fc074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc60800af30 con 0x7fc5fc072960 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.312+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc6080acd20 con 0x7fc61c143470 2024-04-26T21:50:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.313+0000 7fc61b400640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fc5fc072960 0x7fc5fc074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.313+0000 7fc61aa00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc5fc06caa0 0x7fc5fc06ef60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000f60 tx=0x7fc60c0079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.314+0000 7fc61b400640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fc5fc072960 0x7fc5fc074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.317+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== osd.7 v2:172.21.15.204:6808/2746439320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc60800af30 con 0x7fc5fc072960 2024-04-26T21:50:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.330+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 --> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc61c0693c0 con 0x7fc5fc072960 2024-04-26T21:50:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.331+0000 7fc612a00640 1 -- 172.21.15.106:0/4017929297 <== osd.7 v2:172.21.15.204:6808/2746439320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc61c0693c0 con 0x7fc5fc072960 2024-04-26T21:50:50.332 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-26T21:50:50.332 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:50:50.332 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-26T21:50:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fc5fc072960 msgr2=0x7fc5fc074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.204:6808/2746439320,v1:172.21.15.204:6809/2746439320] conn(0x7fc5fc072960 0x7fc5fc074d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc5fc06caa0 msgr2=0x7fc5fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7fc5fc06caa0 0x7fc5fc06ef60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000f60 tx=0x7fc60c0079d0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 msgr2=0x7fc61c1e3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 --2- 172.21.15.106:0/4017929297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc61c143470 0x7fc61c1e3560 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098c0 tx=0x7fc6080046c0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61b400640 1 -- 172.21.15.106:0/4017929297 reap_dead start 2024-04-26T21:50:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 shutdown_connections 2024-04-26T21:50:50.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.332+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 >> 172.21.15.106:0/4017929297 conn(0x7fc61c0d0b50 msgr2=0x7fc61c0b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.333+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 shutdown_connections 2024-04-26T21:50:50.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.333+0000 7fc61be00640 1 -- 172.21.15.106:0/4017929297 wait complete. 2024-04-26T21:50:50.352 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T21:50:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.551+0000 7f66bb800640 1 Processor -- start 2024-04-26T21:50:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.551+0000 7f66bb800640 1 -- start start 2024-04-26T21:50:50.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66bb800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66b415cfd0 0x7f66b406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66bb800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b40ae850 con 0x7f66b415cb70 2024-04-26T21:50:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66bb800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b40aeb30 con 0x7f66b406c770 2024-04-26T21:50:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66bb800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66b40af460 con 0x7f66b415cfd0 2024-04-26T21:50:50.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66b3e00640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f66b415cb70 0x7f66b406ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54972/0 (socket says 172.21.15.106:54972) 2024-04-26T21:50:50.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66b3e00640 1 -- 172.21.15.106:0/3544033268 learned_addr learned my addr 172.21.15.106:0/3544033268 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:50.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.552+0000 7f66b9800640 1 --2- 172.21.15.106:0/3544033268 >> v2:172.21.15.204:3300/0 conn(0x7f66b415cfd0 0x7f66b406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151821941 0 0) 0x7f66b40ae850 con 0x7f66b415cb70 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f669c003520 con 0x7f66b415cb70 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3123629453 0 0) 0x7f669c003520 con 0x7f66b415cb70 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 >> v2:172.21.15.204:3300/0 conn(0x7f66b415cfd0 msgr2=0x7f66b406f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 --2- 172.21.15.106:0/3544033268 >> v2:172.21.15.204:3300/0 conn(0x7f66b415cfd0 0x7f66b406f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 >> v1:172.21.15.204:6789/0 conn(0x7f66b406c770 legacy=0x7f66b406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b40aee10 con 0x7f66b415cb70 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f66a0001670 con 0x7f66b415cb70 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 >> v1:172.21.15.106:6789/0 conn(0x7f66b415cb70 legacy=0x7f66b406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f669c0039b0 0x7f669c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c004800 0x7f669c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 0x7f669c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66b40af460 con 0x7f669c0051c0 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f669c00e2a0 con 0x7f669c004800 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.553+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f669c00e4a0 con 0x7f669c0039b0 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 0x7f669c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b3e00640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c004800 0x7f669c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b8e00640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f669c0039b0 0x7f669c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 -- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f669c0039b0 msgr2=0x7f669c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f669c0039b0 0x7f669c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 -- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c004800 msgr2=0x7f669c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c004800 0x7f669c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.554+0000 7f66b9800640 1 -- 172.21.15.106:0/3544033268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c003520 con 0x7f669c0051c0 2024-04-26T21:50:50.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.555+0000 7f66b9800640 1 -- 172.21.15.106:0/3544033268 reap_dead start 2024-04-26T21:50:50.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66b9800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 0x7f669c00dbd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009fc0 tx=0x7f669c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27faccb4e54077a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac009c10 con 0x7f669c0051c0 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac003990 con 0x7f669c0051c0 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66b3400640 1 -- 172.21.15.106:0/3544033268 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02d070 con 0x7f669c0051c0 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66bb800640 1 -- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 msgr2=0x7f669c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.556+0000 7f66bb800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 0x7f669c00dbd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009fc0 tx=0x7f669c005830 comp rx=0 tx=0).stop 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.557+0000 7f66bb800640 1 -- 172.21.15.106:0/3544033268 shutdown_connections 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.557+0000 7f66bb800640 1 --2- 172.21.15.106:0/3544033268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f669c0051c0 0x7f669c00dbd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.557+0000 7f66bb800640 1 -- 172.21.15.106:0/3544033268 >> 172.21.15.106:0/3544033268 conn(0x7f66b40b2d50 msgr2=0x7f66b40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.557+0000 7f66bb800640 1 -- 172.21.15.106:0/3544033268 shutdown_connections 2024-04-26T21:50:50.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.557+0000 7f66bb800640 1 -- 172.21.15.106:0/3544033268 wait complete. 2024-04-26T21:50:50.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.558+0000 7f66bb800640 1 Processor -- start 2024-04-26T21:50:50.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 -- start start 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c0051c0 0x7f66b41f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66b40b4b80 0x7f66b40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66b415c780 con 0x7f66b40ce260 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66b415c600 con 0x7f669c0051c0 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.559+0000 7f66bb800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66b40af8f0 con 0x7f66b40b4b80 2024-04-26T21:50:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53388/0 (socket says 172.21.15.106:53388) 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 -- 172.21.15.106:0/531226008 learned_addr learned my addr 172.21.15.106:0/531226008 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b8e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c0051c0 0x7f66b41f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b9800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66b40b4b80 0x7f66b40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 -- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66b40b4b80 msgr2=0x7f66b40b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66b40b4b80 0x7f66b40b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 -- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c0051c0 msgr2=0x7f66b41f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c0051c0 0x7f66b41f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.560+0000 7f66b3e00640 1 -- 172.21.15.106:0/531226008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b40c9fb0 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66b3e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e730 tx=0x7f66a000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000ee40 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0018070 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b4152dd0 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cab0 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.561+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41f8cf0 con 0x7f66b40ce260 2024-04-26T21:50:50.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.562+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b40b5540 con 0x7f66b40ce260 2024-04-26T21:50:50.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.563+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f66a000cc50 con 0x7f66b40ce260 2024-04-26T21:50:50.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.564+0000 7f66b0c00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 0x7f669406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.564+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f66a008ba60 con 0x7f66b40ce260 2024-04-26T21:50:50.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.564+0000 7f66b8e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 0x7f669406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.565+0000 7f66b8e00640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 0x7f669406ef60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66a8007fd0 tx=0x7f66a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.568+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66a0059e70 con 0x7f66b40ce260 2024-04-26T21:50:50.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.765+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f66b40d0680 con 0x7f669406caa0 2024-04-26T21:50:50.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.766+0000 7f66b0c00640 1 -- 172.21.15.106:0/531226008 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f66b40d0680 con 0x7f669406caa0 2024-04-26T21:50:50.767 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T21:50:50.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 msgr2=0x7f669406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 0x7f669406ef60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66a8007fd0 tx=0x7f66a8009040 comp rx=0 tx=0).stop 2024-04-26T21:50:50.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 msgr2=0x7f66b4152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e730 tx=0x7f66a000ebf0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 shutdown_connections 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f669406caa0 0x7f669406ef60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66b40b4b80 0x7f66b40b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b40ce260 0x7f66b4152890 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 --2- 172.21.15.106:0/531226008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669c0051c0 0x7f66b41f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 >> 172.21.15.106:0/531226008 conn(0x7f66b40b2d50 msgr2=0x7f66b415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 shutdown_connections 2024-04-26T21:50:50.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.770+0000 7f66bb800640 1 -- 172.21.15.106:0/531226008 wait complete. 2024-04-26T21:50:50.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:50:50.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.978+0000 7f6d57e00640 1 Processor -- start 2024-04-26T21:50:50.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.978+0000 7f6d57e00640 1 -- start start 2024-04-26T21:50:50.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d57e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 0x7f6d500add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d57e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d500cd650 0x7f6d500b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d57e00640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d500af9a0 con 0x7f6d5015cdf0 2024-04-26T21:50:50.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d57e00640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6d500ae2c0 con 0x7f6d500cd650 2024-04-26T21:50:50.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d57e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6d500b05b0 con 0x7f6d5006ce80 2024-04-26T21:50:50.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d55400640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6d5015cdf0 0x7f6d5006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54984/0 (socket says 172.21.15.106:54984) 2024-04-26T21:50:50.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d55400640 1 -- 172.21.15.106:0/3385726616 learned_addr learned my addr 172.21.15.106:0/3385726616 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:50:50.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d55e00640 1 --2- 172.21.15.106:0/3385726616 >> v2:172.21.15.106:3300/0 conn(0x7f6d500cd650 0x7f6d500b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d54a00640 1 --2- 172.21.15.106:0/3385726616 >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 0x7f6d500add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.980+0000 7f6d55e00640 1 -- 172.21.15.106:0/3385726616 >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 msgr2=0x7f6d500add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:50:50.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.981+0000 7f6d55e00640 1 --2- 172.21.15.106:0/3385726616 >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 0x7f6d500add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.981+0000 7f6d4fe00640 1 -- 172.21.15.106:0/3385726616 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847047970 0 0) 0x7f6d500af9a0 con 0x7f6d5015cdf0 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.981+0000 7f6d55e00640 1 -- 172.21.15.106:0/3385726616 >> v1:172.21.15.106:6789/0 conn(0x7f6d5015cdf0 legacy=0x7f6d5006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.981+0000 7f6d55e00640 1 -- 172.21.15.106:0/3385726616 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d500afc80 con 0x7f6d500cd650 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.981+0000 7f6d54a00640 1 --2- 172.21.15.106:0/3385726616 >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 0x7f6d500add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d55e00640 1 --2- 172.21.15.106:0/3385726616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d500b3d10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6d500d1d00 tx=0x7f6d4802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c94c696e4a21236 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d4fe00640 1 -- 172.21.15.106:0/3385726616 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007740 con 0x7f6d500cd650 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d4fe00640 1 -- 172.21.15.106:0/3385726616 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d480078e0 con 0x7f6d500cd650 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d4fe00640 1 -- 172.21.15.106:0/3385726616 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007cb0 con 0x7f6d500cd650 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d57e00640 1 -- 172.21.15.106:0/3385726616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 msgr2=0x7f6d500b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.982+0000 7f6d57e00640 1 --2- 172.21.15.106:0/3385726616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d500b3d10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6d500d1d00 tx=0x7f6d4802a680 comp rx=0 tx=0).stop 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 -- 172.21.15.106:0/3385726616 shutdown_connections 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 --2- 172.21.15.106:0/3385726616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d500b3d10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 --2- 172.21.15.106:0/3385726616 >> v2:172.21.15.204:3300/0 conn(0x7f6d5006ce80 0x7f6d500add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 -- 172.21.15.106:0/3385726616 >> 172.21.15.106:0/3385726616 conn(0x7f6d500b2d50 msgr2=0x7f6d500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 -- 172.21.15.106:0/3385726616 shutdown_connections 2024-04-26T21:50:50.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.983+0000 7f6d57e00640 1 -- 172.21.15.106:0/3385726616 wait complete. 2024-04-26T21:50:50.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.984+0000 7f6d57e00640 1 Processor -- start 2024-04-26T21:50:50.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.984+0000 7f6d57e00640 1 -- start start 2024-04-26T21:50:50.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d5006ce80 0x7f6d5014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d5015cdf0 0x7f6d500b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5015d1d0 con 0x7f6d500cd650 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d500b0240 con 0x7f6d5015cdf0 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d57e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5015d350 con 0x7f6d5006ce80 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d54a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d54a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53398/0 (socket says 172.21.15.106:53398) 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.985+0000 7f6d54a00640 1 -- 172.21.15.106:0/2176147418 learned_addr learned my addr 172.21.15.106:0/2176147418 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:50:50.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 -- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d5006ce80 msgr2=0x7f6d5014f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d5006ce80 0x7f6d5014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 -- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d5015cdf0 msgr2=0x7f6d500b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d5015cdf0 0x7f6d500b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 -- 172.21.15.106:0/2176147418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5009c7b0 con 0x7f6d500cd650 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d54a00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c7f0 tx=0x7f6d3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c015070 con 0x7f6d500cd650 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c007a30 con 0x7f6d500cd650 2024-04-26T21:50:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c007e00 con 0x7f6d500cd650 2024-04-26T21:50:50.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5009ca90 con 0x7f6d500cd650 2024-04-26T21:50:50.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.986+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5009d330 con 0x7f6d500cd650 2024-04-26T21:50:50.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.987+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5009cc60 con 0x7f6d500cd650 2024-04-26T21:50:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.988+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6d3c018070 con 0x7f6d500cd650 2024-04-26T21:50:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.989+0000 7f6d4d600640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 0x7f6d3006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.989+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f6d3c08bc00 con 0x7f6d500cd650 2024-04-26T21:50:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.989+0000 7f6d55400640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 0x7f6d3006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.990+0000 7f6d55400640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 0x7f6d3006ef60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007fd0 tx=0x7f6d44007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:50.993+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d3c092080 con 0x7f6d500cd650 2024-04-26T21:50:51.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.177+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d50069540 con 0x7f6d3006caa0 2024-04-26T21:50:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.178+0000 7f6d4d600640 1 -- 172.21.15.106:0/2176147418 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13429 (secure 0 0 0) 0x7f6d50069540 con 0x7f6d3006caa0 2024-04-26T21:50:51.179 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:50:51.180 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.180+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 msgr2=0x7f6d3006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.180+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 0x7f6d3006ef60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007fd0 tx=0x7f6d44007960 comp rx=0 tx=0).stop 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.180+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 msgr2=0x7f6d501580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.180+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c7f0 tx=0x7f6d3c00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 shutdown_connections 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d5015cdf0 0x7f6d500b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d500cd650 0x7f6d501580d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f6d3006caa0 0x7f6d3006ef60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 --2- 172.21.15.106:0/2176147418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d5006ce80 0x7f6d5014f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 >> 172.21.15.106:0/2176147418 conn(0x7f6d500b2d50 msgr2=0x7f6d500d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:51.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 shutdown_connections 2024-04-26T21:50:51.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:50:51.181+0000 7f6d57e00640 1 -- 172.21.15.106:0/2176147418 wait complete. 2024-04-26T21:50:51.198 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-04-26T21:50:49.188609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238660,"kb_used_data":5832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736060,"statfs":{"total":773094113280,"available":772849725440,"internally_reserved":0,"allocated":5971968,"data_stored":3140112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002462"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:50:40.073663+0000","last_change":"2024-04-26T21:45:46.741028+0000","last_active":"2024-04-26T21:50:40.073663+0000","last_peered":"2024-04-26T21:50:40.073663+0000","last_clean":"2024-04-26T21:50:40.073663+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:50:40.073663+0000","last_undegraded":"2024-04-26T21:50:40.073663+0000","last_fullsized":"2024-04-26T21:50:40.073663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:45:45.474677+0000","last_clean_scrub_stamp":"2024-04-26T21:45:45.474677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:43.090845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:50:51.200 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T21:45:45.474677+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=45, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=50, tm_sec=44, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T21:50:51.200 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T21:51:11.202 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:51:11.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.412+0000 7ffac8800640 1 Processor -- start 2024-04-26T21:51:11.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.412+0000 7ffac8800640 1 -- start start 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.413+0000 7ffac8800640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffac01598a0 0x7ffac01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac8800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffac006c770 0x7ffac006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac8800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac00b5fb0 con 0x7ffac015f740 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac8800640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ffac0159c80 con 0x7ffac01598a0 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac8800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ffac006d110 con 0x7ffac006c770 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffac01598a0 0x7ffac01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffac01598a0 0x7ffac01638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35262/0 (socket says 172.21.15.106:35262) 2024-04-26T21:51:11.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 -- 172.21.15.106:0/2388705540 learned_addr learned my addr 172.21.15.106:0/2388705540 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:51:11.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 -- 172.21.15.106:0/2388705540 >> v2:172.21.15.204:3300/0 conn(0x7ffac006c770 msgr2=0x7ffac006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:11.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 --2- 172.21.15.106:0/2388705540 >> v2:172.21.15.204:3300/0 conn(0x7ffac006c770 0x7ffac006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 -- 172.21.15.106:0/2388705540 >> v1:172.21.15.204:6789/0 conn(0x7ffac015f740 legacy=0x7ffac015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.414+0000 7ffac5e00640 1 -- 172.21.15.106:0/2388705540 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac0160370 con 0x7ffac01598a0 2024-04-26T21:51:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.415+0000 7ffac5e00640 1 --2- 172.21.15.106:0/2388705540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac01598a0 0x7ffac01638b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffac00b3900 tx=0x7ffab402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e1167ec1ca128 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac4a00640 1 -- 172.21.15.106:0/2388705540 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4037070 con 0x7ffac01598a0 2024-04-26T21:51:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac4a00640 1 -- 172.21.15.106:0/2388705540 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab4032440 con 0x7ffac01598a0 2024-04-26T21:51:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac4a00640 1 -- 172.21.15.106:0/2388705540 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4032810 con 0x7ffac01598a0 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac8800640 1 -- 172.21.15.106:0/2388705540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac01598a0 msgr2=0x7ffac01638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac8800640 1 --2- 172.21.15.106:0/2388705540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac01598a0 0x7ffac01638b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffac00b3900 tx=0x7ffab402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac8800640 1 -- 172.21.15.106:0/2388705540 shutdown_connections 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac8800640 1 --2- 172.21.15.106:0/2388705540 >> v2:172.21.15.204:3300/0 conn(0x7ffac006c770 0x7ffac006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.416+0000 7ffac8800640 1 --2- 172.21.15.106:0/2388705540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac01598a0 0x7ffac01638b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.417+0000 7ffac8800640 1 -- 172.21.15.106:0/2388705540 >> 172.21.15.106:0/2388705540 conn(0x7ffac00b2d50 msgr2=0x7ffac00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:11.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.417+0000 7ffac8800640 1 -- 172.21.15.106:0/2388705540 shutdown_connections 2024-04-26T21:51:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.417+0000 7ffac8800640 1 -- 172.21.15.106:0/2388705540 wait complete. 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.418+0000 7ffac8800640 1 Processor -- start 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.418+0000 7ffac8800640 1 -- start start 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac01598a0 0x7ffac00ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffac015f740 0x7ffac00ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0163390 con 0x7ffac006c770 2024-04-26T21:51:11.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01630b0 con 0x7ffac01598a0 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.419+0000 7ffac8800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffac0160650 con 0x7ffac015f740 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35266/0 (socket says 172.21.15.106:35266) 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 -- 172.21.15.106:0/4013866601 learned_addr learned my addr 172.21.15.106:0/4013866601 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 -- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffac015f740 msgr2=0x7ffac00ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffac015f740 0x7ffac00ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 -- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac01598a0 msgr2=0x7ffac00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac01598a0 0x7ffac00ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 -- 172.21.15.106:0/4013866601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac00aeb70 con 0x7ffac006c770 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.420+0000 7ffac5e00640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000c00 tx=0x7ffab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.421+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4041070 con 0x7ffac006c770 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.421+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab403c870 con 0x7ffac006c770 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.421+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac00aee50 con 0x7ffac006c770 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.421+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4037040 con 0x7ffac006c770 2024-04-26T21:51:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.421+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac00af410 con 0x7ffac006c770 2024-04-26T21:51:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.422+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac0069540 con 0x7ffac006c770 2024-04-26T21:51:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.423+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ffab4044070 con 0x7ffac006c770 2024-04-26T21:51:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.424+0000 7ffaba000640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 0x7ffaa406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.424+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ffab40ac880 con 0x7ffac006c770 2024-04-26T21:51:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.424+0000 7ffac5400640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 0x7ffaa406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.425+0000 7ffac5400640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 0x7ffaa406ef90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffac0147f60 tx=0x7ffab0006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.429+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffab407adb0 con 0x7ffac006c770 2024-04-26T21:51:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.626+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 --> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffac0162630 con 0x7ffaa406cad0 2024-04-26T21:51:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.628+0000 7ffaba000640 1 -- 172.21.15.106:0/4013866601 <== mgr.4107 v2:172.21.15.106:6832/4027448399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7ffac0162630 con 0x7ffaa406cad0 2024-04-26T21:51:11.629 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:51:11.629 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-26T21:51:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 msgr2=0x7ffaa406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 0x7ffaa406ef90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffac0147f60 tx=0x7ffab0006bf0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 msgr2=0x7ffac00afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000c00 tx=0x7ffab40046c0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 shutdown_connections 2024-04-26T21:51:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7ffaa406cad0 0x7ffaa406ef90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffac015f740 0x7ffac00ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac01598a0 0x7ffac00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 --2- 172.21.15.106:0/4013866601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffac006c770 0x7ffac00afa70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 >> 172.21.15.106:0/4013866601 conn(0x7ffac00b2d50 msgr2=0x7ffac00d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 shutdown_connections 2024-04-26T21:51:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.631+0000 7ffac8800640 1 -- 172.21.15.106:0/4013866601 wait complete. 2024-04-26T21:51:11.650 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-04-26T21:51:11.194537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238640,"kb_used_data":5832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736080,"statfs":{"total":773094113280,"available":772849745920,"internally_reserved":0,"allocated":5971968,"data_stored":3140112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003451"},"pg_stats":[{"pgid":"1.0","version":"50'119","reported_seq":243,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:50:51.241284+0000","last_change":"2024-04-26T21:50:51.241359+0000","last_active":"2024-04-26T21:50:51.241284+0000","last_peered":"2024-04-26T21:50:51.241284+0000","last_clean":"2024-04-26T21:50:51.241284+0000","last_became_active":"2024-04-26T21:45:46.740302+0000","last_became_peered":"2024-04-26T21:45:46.740302+0000","last_unstale":"2024-04-26T21:50:51.241284+0000","last_undegraded":"2024-04-26T21:50:51.241284+0000","last_fullsized":"2024-04-26T21:50:51.241284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'119","last_scrub_stamp":"2024-04-26T21:50:51.241059+0000","last_deep_scrub":"50'119","last_deep_scrub_stamp":"2024-04-26T21:50:51.241059+0000","last_clean_scrub_stamp":"2024-04-26T21:50:51.241059+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:06.671204+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":244922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1191936,"data_stored":835290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:51:11.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T21:51:11.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.841+0000 7f0c03200640 1 Processor -- start 2024-04-26T21:51:11.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.842+0000 7f0c03200640 1 -- start start 2024-04-26T21:51:11.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0c04071e80 0x7f0c040b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c040ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c04072260 0x7f0c040726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0406e320 con 0x7f0c04071e80 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0406e600 con 0x7f0c04072260 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.843+0000 7f0c03200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0406e8e0 con 0x7f0c040b4ed0 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0c04071e80 0x7f0c040b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0c04071e80 0x7f0c040b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35282/0 (socket says 172.21.15.106:35282) 2024-04-26T21:51:11.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 -- 172.21.15.106:0/3778879368 learned_addr learned my addr 172.21.15.106:0/3778879368 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:51:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 -- 172.21.15.106:0/3778879368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 msgr2=0x7f0c040ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:11.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3778879368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c040ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 -- 172.21.15.106:0/3778879368 >> v2:172.21.15.204:3300/0 conn(0x7f0c04072260 msgr2=0x7f0c040726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3778879368 >> v2:172.21.15.204:3300/0 conn(0x7f0c04072260 0x7f0c040726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.844+0000 7f0c01e00640 1 -- 172.21.15.106:0/3778879368 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0406ebc0 con 0x7f0c04071e80 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.845+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3778879368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040b3ee0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0c04073d50 tx=0x7f0bf402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618aac552846fbde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.845+0000 7f0c00a00640 1 -- 172.21.15.106:0/3778879368 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4037070 con 0x7f0c04071e80 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.846+0000 7f0c00a00640 1 -- 172.21.15.106:0/3778879368 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf4032440 con 0x7f0c04071e80 2024-04-26T21:51:11.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.846+0000 7f0c00a00640 1 -- 172.21.15.106:0/3778879368 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4032810 con 0x7f0c04071e80 2024-04-26T21:51:11.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.846+0000 7f0c03200640 1 -- 172.21.15.106:0/3778879368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 msgr2=0x7f0c040b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.846+0000 7f0c03200640 1 --2- 172.21.15.106:0/3778879368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040b3ee0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0c04073d50 tx=0x7f0bf402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 -- 172.21.15.106:0/3778879368 shutdown_connections 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 --2- 172.21.15.106:0/3778879368 >> v2:172.21.15.204:3300/0 conn(0x7f0c04072260 0x7f0c040726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 --2- 172.21.15.106:0/3778879368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c040ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 --2- 172.21.15.106:0/3778879368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040b3ee0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 -- 172.21.15.106:0/3778879368 >> 172.21.15.106:0/3778879368 conn(0x7f0c040b2900 msgr2=0x7f0c040b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 -- 172.21.15.106:0/3778879368 shutdown_connections 2024-04-26T21:51:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.847+0000 7f0c03200640 1 -- 172.21.15.106:0/3778879368 wait complete. 2024-04-26T21:51:11.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.849+0000 7f0c03200640 1 Processor -- start 2024-04-26T21:51:11.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.849+0000 7f0c03200640 1 -- start start 2024-04-26T21:51:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c04072260 0x7f0c04144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c04161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c040b5ff0 con 0x7f0c04071e80 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0415f4d0 con 0x7f0c04072260 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c03200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0406f0e0 con 0x7f0c040b4ed0 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35298/0 (socket says 172.21.15.106:35298) 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 -- 172.21.15.106:0/3594529684 learned_addr learned my addr 172.21.15.106:0/3594529684 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 -- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 msgr2=0x7f0c04161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c04161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 -- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c04072260 msgr2=0x7f0c04144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c04072260 0x7f0c04144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.850+0000 7f0c01e00640 1 -- 172.21.15.106:0/3594529684 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c04161e80 con 0x7f0c04071e80 2024-04-26T21:51:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0c01e00640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402a220 tx=0x7f0bf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4041070 con 0x7f0c04071e80 2024-04-26T21:51:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf403c890 con 0x7f0c04071e80 2024-04-26T21:51:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4037040 con 0x7f0c04071e80 2024-04-26T21:51:11.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c04162160 con 0x7f0c04071e80 2024-04-26T21:51:11.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.851+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c04152f70 con 0x7f0c04071e80 2024-04-26T21:51:11.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.852+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c040693c0 con 0x7f0c04071e80 2024-04-26T21:51:11.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.853+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f0bf4044070 con 0x7f0c04071e80 2024-04-26T21:51:11.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.854+0000 7f0bfa000640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 0x7f0be006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.854+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f0bf40ac970 con 0x7f0c04071e80 2024-04-26T21:51:11.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.854+0000 7f0c01400640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 0x7f0be006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.855+0000 7f0c01400640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 0x7f0be006ef60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c040b01f0 tx=0x7f0bec006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:11.858+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bf407af30 con 0x7f0c04071e80 2024-04-26T21:51:12.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.061+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f0c040ccc50 con 0x7f0c04071e80 2024-04-26T21:51:12.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.066+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f0bf407a830 con 0x7f0c04071e80 2024-04-26T21:51:12.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.067+0000 7f0bfa000640 1 -- 172.21.15.106:0/3594529684 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0bf40b2080 con 0x7f0c04071e80 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 msgr2=0x7f0be006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 0x7f0be006ef60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c040b01f0 tx=0x7f0bec006bf0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 msgr2=0x7f0c040ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402a220 tx=0x7f0bf40046c0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 shutdown_connections 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f0be006caa0 0x7f0be006ef60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c040b4ed0 0x7f0c04161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c04072260 0x7f0c04144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 --2- 172.21.15.106:0/3594529684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c04071e80 0x7f0c040ce470 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.069+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 >> 172.21.15.106:0/3594529684 conn(0x7f0c040b2900 msgr2=0x7f0c0415d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.070+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 shutdown_connections 2024-04-26T21:51:12.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.070+0000 7f0c03200640 1 -- 172.21.15.106:0/3594529684 wait complete. 2024-04-26T21:51:12.089 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:51:12.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:51:12.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.281+0000 7f5e47200640 1 Processor -- start 2024-04-26T21:51:12.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.282+0000 7f5e47200640 1 -- start start 2024-04-26T21:51:12.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.283+0000 7f5e47200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5e48146680 0x7f5e480b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.283+0000 7f5e47200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e480d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.283+0000 7f5e47200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e48146a60 con 0x7f5e48160930 2024-04-26T21:51:12.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.283+0000 7f5e47200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4806e970 con 0x7f5e48146680 2024-04-26T21:51:12.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.283+0000 7f5e47200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e480d17f0 con 0x7f5e480d0e70 2024-04-26T21:51:12.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e46800640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5e48160930 0x7f5e48160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37706/0 (socket says 172.21.15.106:37706) 2024-04-26T21:51:12.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e46800640 1 -- 172.21.15.106:0/1991054089 learned_addr learned my addr 172.21.15.106:0/1991054089 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-26T21:51:12.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592635546 0 0) 0x7f5e48146a60 con 0x7f5e48160930 2024-04-26T21:51:12.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e45400640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e480d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e2c003520 con 0x7f5e48160930 2024-04-26T21:51:12.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e45e00640 1 --2- 172.21.15.106:0/1991054089 >> v2:172.21.15.204:3300/0 conn(0x7f5e48146680 0x7f5e480b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.284+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 652782625 0 0) 0x7f5e2c003520 con 0x7f5e48160930 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 msgr2=0x7f5e480d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e480d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 >> v2:172.21.15.204:3300/0 conn(0x7f5e48146680 msgr2=0x7f5e480b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 --2- 172.21.15.106:0/1991054089 >> v2:172.21.15.204:3300/0 conn(0x7f5e48146680 0x7f5e480b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4806dd60 con 0x7f5e48160930 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 952049768 0 0) 0x7f5e400019f0 con 0x7f5e48160930 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 >> v1:172.21.15.106:6789/0 conn(0x7f5e48160930 legacy=0x7f5e48160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c0039b0 0x7f5e2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c004800 0x7f5e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480d17f0 con 0x7f5e2c0039b0 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c00b9c0 con 0x7f5e2c004800 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c00bb40 con 0x7f5e480d0e70 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e45e00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c0039b0 0x7f5e2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.285+0000 7f5e46800640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45400640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c004800 0x7f5e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45e00640 1 -- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 msgr2=0x7f5e2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:12.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45e00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e480d0e70 0x7f5e2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45e00640 1 -- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c004800 msgr2=0x7f5e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:51:12.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45e00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c004800 0x7f5e2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.286+0000 7f5e45e00640 1 -- 172.21.15.106:0/1991054089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c003520 con 0x7f5e2c0039b0 2024-04-26T21:51:12.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.287+0000 7f5e45e00640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c0039b0 0x7f5e2c003dd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800ba30 tx=0x7f5e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e4ad61aed6bf9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38030070 con 0x7f5e2c0039b0 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5e3802ae10 con 0x7f5e2c0039b0 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e44a00640 1 -- 172.21.15.106:0/1991054089 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800b600 con 0x7f5e2c0039b0 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e47200640 1 -- 172.21.15.106:0/1991054089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c0039b0 msgr2=0x7f5e2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e47200640 1 --2- 172.21.15.106:0/1991054089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c0039b0 0x7f5e2c003dd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800ba30 tx=0x7f5e2c005830 comp rx=0 tx=0).stop 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e46800640 1 -- 172.21.15.106:0/1991054089 reap_dead start 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e47200640 1 -- 172.21.15.106:0/1991054089 shutdown_connections 2024-04-26T21:51:12.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e47200640 1 -- 172.21.15.106:0/1991054089 >> 172.21.15.106:0/1991054089 conn(0x7f5e4806c740 msgr2=0x7f5e4806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.288+0000 7f5e47200640 1 -- 172.21.15.106:0/1991054089 shutdown_connections 2024-04-26T21:51:12.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.289+0000 7f5e47200640 1 -- 172.21.15.106:0/1991054089 wait complete. 2024-04-26T21:51:12.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.290+0000 7f5e47200640 1 Processor -- start 2024-04-26T21:51:12.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.290+0000 7f5e47200640 1 -- start start 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c0039b0 0x7f5e4815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e4813f9f0 0x7f5e4813eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4806e600 con 0x7f5e4815fd20 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4806e320 con 0x7f5e2c0039b0 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.291+0000 7f5e47200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e480cd6a0 con 0x7f5e4813f9f0 2024-04-26T21:51:12.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35324/0 (socket says 172.21.15.106:35324) 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 -- 172.21.15.106:0/294948525 learned_addr learned my addr 172.21.15.106:0/294948525 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 -- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e4813f9f0 msgr2=0x7f5e4813eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e4813f9f0 0x7f5e4813eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 -- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c0039b0 msgr2=0x7f5e4815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45e00640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c0039b0 0x7f5e4815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c0039b0 0x7f5e4815e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 -- 172.21.15.106:0/294948525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4815ec80 con 0x7f5e4815fd20 2024-04-26T21:51:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.292+0000 7f5e45400640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5e34011910 tx=0x7f5e34011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.293+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400fa70 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.293+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5e3400fc10 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.293+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34014440 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.293+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4815ef60 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.293+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4815f520 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.294+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e480693c0 con 0x7f5e4815fd20 2024-04-26T21:51:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.295+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5e34012040 con 0x7f5e4815fd20 2024-04-26T21:51:12.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.296+0000 7f5e3e000640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 0x7f5e2806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.296+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f5e3408c210 con 0x7f5e4815fd20 2024-04-26T21:51:12.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.296+0000 7f5e45e00640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 0x7f5e2806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.297+0000 7f5e45e00640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 0x7f5e2806ef90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800ba00 tx=0x7f5e38030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.303+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e34019030 con 0x7f5e4815fd20 2024-04-26T21:51:12.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.543+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e480cfff0 con 0x7f5e4815fd20 2024-04-26T21:51:12.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.544+0000 7f5e3e000640 1 -- 172.21.15.106:0/294948525 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3878 (secure 0 0 0) 0x7f5e3401c360 con 0x7f5e4815fd20 2024-04-26T21:51:12.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-26T21:51:12.546 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/932826526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":932826526},{"type":"v1","addr":"172.21.15.204:6834","nonce":932826526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2516771969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2516771969},{"type":"v1","addr":"172.21.15.204:6835","nonce":2516771969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5775,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/4113399345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":4113399345},{"type":"v1","addr":"172.21.15.106:6835","nonce":4113399345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4104403473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4104403473},{"type":"v1","addr":"172.21.15.106:6837","nonce":4104403473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-04-26T21:51:12.546 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 28 2024-04-26T21:51:12.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 msgr2=0x7f5e2806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 0x7f5e2806ef90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800ba00 tx=0x7f5e38030000 comp rx=0 tx=0).stop 2024-04-26T21:51:12.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 msgr2=0x7f5e4813e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5e34011910 tx=0x7f5e34011dd0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 shutdown_connections 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:6832/4027448399,v1:172.21.15.106:6833/4027448399] conn(0x7f5e2806cad0 0x7f5e2806ef90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e4813f9f0 0x7f5e4813eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4815fd20 0x7f5e4813e570 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 --2- 172.21.15.106:0/294948525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e2c0039b0 0x7f5e4815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 >> 172.21.15.106:0/294948525 conn(0x7f5e4806c740 msgr2=0x7f5e480b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 shutdown_connections 2024-04-26T21:51:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-26T21:51:12.547+0000 7f5e47200640 1 -- 172.21.15.106:0/294948525 wait complete. 2024-04-26T21:51:12.567 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T21:51:12.568 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T21:51:12.568 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:51:12.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.589 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:51:12.589 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:51:12.589 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.659 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:51:12.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:51:12.660 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.681 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:51:12.692 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:51:12.692 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.745 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:51:12.745 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T21:51:12.745 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T21:51:12.745 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.828 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T21:51:12.828 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T21:51:12.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.904 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T21:51:12.905 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T21:51:12.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:12.983 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T21:51:12.983 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T21:51:12.983 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.060 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T21:51:13.060 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.140 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T21:51:13.140 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T21:51:13.140 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.193 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T21:51:13.193 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T21:51:13.193 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.258 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T21:51:13.258 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T21:51:13.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.321 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T21:51:13.321 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T21:51:13.322 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T21:51:13.322 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:13.408 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T21:51:13.408 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T21:51:13.408 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:16.436 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:16.444 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:17.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-26T21:51:19.510 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T21:51:19.510 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T21:51:19.511 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T21:51:19.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:19.563 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T21:51:19.564 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T21:51:19.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:19.595 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T21:51:19.596 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T21:51:19.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:51:19.676 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T21:51:19.676 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T21:51:19.677 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-26T21:51:19.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:51:19.787 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T21:51:20.008 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:51:20.008 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T21:51:20.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:51:20.145 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T21:51:20.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:51:20.295 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T21:51:20.426 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:51:20.426 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T21:51:20.667 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:51:20.667 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T21:51:20.811 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:51:20.811 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T21:51:20.967 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:51:20.968 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T21:51:21.132 INFO:tasks.ceph:Archiving mon data... 2024-04-26T21:51:21.133 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/data/mon.a.tgz 2024-04-26T21:51:21.133 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-26T21:51:21.191 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.FaPWOJysEz 2024-04-26T21:51:21.191 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FaPWOJysEz 2024-04-26T21:51:21.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.FaPWOJysEz 2024-04-26T21:51:21.538 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.FaPWOJysEz is 434KB 2024-04-26T21:51:21.646 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.FaPWOJysEz 2024-04-26T21:51:21.703 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/data/mon.b.tgz 2024-04-26T21:51:21.703 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-26T21:51:21.761 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.pXzkVuWMTy 2024-04-26T21:51:21.761 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pXzkVuWMTy 2024-04-26T21:51:21.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.pXzkVuWMTy 2024-04-26T21:51:22.125 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.pXzkVuWMTy is 583KB 2024-04-26T21:51:22.238 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.pXzkVuWMTy 2024-04-26T21:51:22.293 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/data/mon.c.tgz 2024-04-26T21:51:22.293 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-26T21:51:22.340 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.r8SOHoguVq 2024-04-26T21:51:22.341 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.r8SOHoguVq 2024-04-26T21:51:22.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-26T21:51:22.544 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.r8SOHoguVq 2024-04-26T21:51:22.684 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.r8SOHoguVq is 582KB 2024-04-26T21:51:22.796 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.r8SOHoguVq 2024-04-26T21:51:22.851 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T21:51:22.852 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:51:22.859 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:51:23.035 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:51:23.092 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:51:23.092 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T21:51:23.094 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi106/crash 2024-04-26T21:51:23.095 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:51:23.176 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi204/crash 2024-04-26T21:51:23.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:51:23.263 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T21:51:23.264 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:23.271 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:23.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:23.469 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:51:23.557 INFO:tasks.ceph:Compressing logs... 2024-04-26T21:51:23.558 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:51:23.565 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:51:23.641 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-04-26T21:51:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48393.log 2024-04-26T21:51:23.643 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33218.log 2024-04-26T21:51:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52271.log 2024-04-26T21:51:23.644 INFO:teuthology.orchestra.run.smithi106.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-04-26T21:51:23.644 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:51:23.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38930.log 2024-04-26T21:51:23.645 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40500.log 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33218.log.gz 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.40500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:51:23.648 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.40500.log.gz 2024-04-26T21:51:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-04-26T21:51:23.649 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52271.log.gz 2024-04-26T21:51:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38930.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38930.log.gz 2024-04-26T21:51:23.650 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53610.log 2024-04-26T21:51:23.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2024-04-26T21:51:23.650 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-04-26T21:51:23.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50129.log 2024-04-26T21:51:23.650 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53610.log.gz 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50297.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33787.log 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46664.log 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50129.log.gz 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-04-26T21:51:23.653 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33787.log.gz 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T21:51:23.654 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-04-26T21:51:23.656 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-04-26T21:51:23.656 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T21:51:23.656 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33469.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T21:51:23.656 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46664.log.gz 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50725.log 2024-04-26T21:51:23.658 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33309.log 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33429.log 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33309.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33309.log.gz 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33429.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33429.log.gz 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T21:51:23.664 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33349.log 2024-04-26T21:51:23.666 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53736.log 2024-04-26T21:51:23.666 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.50725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38290.log 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53736.log: 89.7% -- replaced with /var/log/ceph/ceph-client.0.50725.log.gz 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53736.log.gz 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45846.log 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38290.log.gz 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52355.log 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2024-04-26T21:51:23.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2024-04-26T21:51:23.668 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45846.log.gz 2024-04-26T21:51:23.668 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-04-26T21:51:23.668 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52355.log.gz 2024-04-26T21:51:23.668 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47150.log 2024-04-26T21:51:23.668 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33349.log: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33349.log.gz 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33389.log 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33178.log 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33389.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33389.log.gz 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33178.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33178.log.gz 2024-04-26T21:51:23.672 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39289.log 2024-04-26T21:51:23.673 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47150.log.gz 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-04-26T21:51:23.674 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45968.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2024-04-26T21:51:23.675 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-04-26T21:51:23.675 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2024-04-26T21:51:23.675 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-04-26T21:51:23.675 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46054.log 2024-04-26T21:51:23.675 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-04-26T21:51:23.678 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T21:51:23.678 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.39289.log: /var/log/ceph/ceph-mds.d.log: 90.6% -- replaced with /var/log/ceph/ceph-client.1.39289.log.gz 2024-04-26T21:51:23.678 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45346.log 2024-04-26T21:51:23.679 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2024-04-26T21:51:23.680 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42817.log 2024-04-26T21:51:23.680 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2024-04-26T21:51:23.680 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38888.log 2024-04-26T21:51:23.681 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41148.log 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45346.log.gz 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42817.log.gz 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38888.log.gz 2024-04-26T21:51:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51108.log 2024-04-26T21:51:23.683 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.1.41148.log: 90.8% -- replaced with /var/log/ceph/ceph-client.1.41148.log.gz 2024-04-26T21:51:23.684 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39745.log 2024-04-26T21:51:23.684 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-04-26T21:51:23.684 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47567.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:51:23.685 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2024-04-26T21:51:23.685 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51108.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51108.log.gz 2024-04-26T21:51:23.685 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2024-04-26T21:51:23.685 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-04-26T21:51:23.687 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50339.log 2024-04-26T21:51:23.687 INFO:teuthology.orchestra.run.smithi106.stderr: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:51:23.687 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2024-04-26T21:51:23.688 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50948.log 2024-04-26T21:51:23.688 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-04-26T21:51:23.689 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40442.log 2024-04-26T21:51:23.689 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2024-04-26T21:51:23.690 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50948.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 2024-04-26T21:51:23.690 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46012.log 2024-04-26T21:51:23.690 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.3.39745.log: 91.6% -- replaced with /var/log/ceph/ceph-client.3.39745.log.gz 2024-04-26T21:51:23.691 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-04-26T21:51:23.691 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40442.log.gz 2024-04-26T21:51:23.692 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53904.log 2024-04-26T21:51:23.692 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46012.log.gz 2024-04-26T21:51:23.693 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33687.log 2024-04-26T21:51:23.693 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-04-26T21:51:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53298.log 2024-04-26T21:51:23.695 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53904.log.gz 2024-04-26T21:51:23.695 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33687.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T21:51:23.695 INFO:teuthology.orchestra.run.smithi106.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33687.log.gz 2024-04-26T21:51:23.696 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42649.log 2024-04-26T21:51:23.696 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53298.log.gz 2024-04-26T21:51:23.697 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40204.log 2024-04-26T21:51:23.698 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43374.log 2024-04-26T21:51:23.698 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42649.log.gz 2024-04-26T21:51:23.699 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40204.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40204.log.gz 2024-04-26T21:51:23.699 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2024-04-26T21:51:23.700 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53568.log 2024-04-26T21:51:23.701 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43374.log.gz 2024-04-26T21:51:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51334.log 2024-04-26T21:51:23.701 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2024-04-26T21:51:23.703 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53568.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53568.log.gz 2024-04-26T21:51:23.703 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39868.log 2024-04-26T21:51:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39518.log 2024-04-26T21:51:23.704 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51334.log.gz 2024-04-26T21:51:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43949.log 2024-04-26T21:51:23.705 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39868.log.gz 2024-04-26T21:51:23.706 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-04-26T21:51:23.706 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39518.log.gz 2024-04-26T21:51:23.707 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39867.log 2024-04-26T21:51:23.707 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43949.log.gz 2024-04-26T21:51:23.708 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48435.log 2024-04-26T21:51:23.708 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-04-26T21:51:23.709 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-04-26T21:51:23.710 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39867.log.gz 2024-04-26T21:51:23.710 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2024-04-26T21:51:23.711 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48435.log.gz 2024-04-26T21:51:23.712 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41132.log 2024-04-26T21:51:23.712 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-04-26T21:51:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50906.log 2024-04-26T21:51:23.713 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2024-04-26T21:51:23.714 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41132.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:51:23.714 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41132.log.gz 2024-04-26T21:51:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-04-26T21:51:23.715 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50906.log.gz 2024-04-26T21:51:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-04-26T21:51:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52439.log 2024-04-26T21:51:23.718 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-04-26T21:51:23.718 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:51:23.719 INFO:teuthology.orchestra.run.smithi204.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T21:51:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-04-26T21:51:23.720 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz 2024-04-26T21:51:23.720 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-04-26T21:51:23.720 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52439.log.gz 2024-04-26T21:51:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2024-04-26T21:51:23.721 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-04-26T21:51:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39865.log 2024-04-26T21:51:23.722 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-04-26T21:51:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38674.log 2024-04-26T21:51:23.723 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2024-04-26T21:51:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-04-26T21:51:23.724 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39865.log.gz 2024-04-26T21:51:23.725 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52838.log 2024-04-26T21:51:23.725 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38674.log.gz 2024-04-26T21:51:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38972.log 2024-04-26T21:51:23.726 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-04-26T21:51:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38508.log 2024-04-26T21:51:23.727 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52838.log.gz 2024-04-26T21:51:23.728 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43195.log 2024-04-26T21:51:23.729 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38972.log.gz 2024-04-26T21:51:23.729 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42141.log 2024-04-26T21:51:23.730 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38508.log.gz 2024-04-26T21:51:23.731 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-04-26T21:51:23.731 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43195.log.gz 2024-04-26T21:51:23.732 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.2.42141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51580.log 2024-04-26T21:51:23.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53986.log 2024-04-26T21:51:23.733 INFO:teuthology.orchestra.run.smithi106.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.2.42141.log.gz 2024-04-26T21:51:23.733 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-04-26T21:51:23.734 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44496.log 2024-04-26T21:51:23.734 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51580.log.gz 2024-04-26T21:51:23.735 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-04-26T21:51:23.735 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53986.log.gz 2024-04-26T21:51:23.736 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47809.log 2024-04-26T21:51:23.736 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2024-04-26T21:51:23.737 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49862.log 2024-04-26T21:51:23.737 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-04-26T21:51:23.738 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38758.log 2024-04-26T21:51:23.739 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47809.log.gz 2024-04-26T21:51:23.739 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46874.log 2024-04-26T21:51:23.740 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49862.log.gz 2024-04-26T21:51:23.741 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38758.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T21:51:23.741 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38758.log.gz 2024-04-26T21:51:23.742 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51706.log 2024-04-26T21:51:23.742 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46874.log.gz 2024-04-26T21:51:23.743 INFO:teuthology.orchestra.run.smithi204.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T21:51:23.743 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-04-26T21:51:23.744 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-04-26T21:51:23.744 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51706.log.gz 2024-04-26T21:51:23.745 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53118.log 2024-04-26T21:51:23.745 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-04-26T21:51:23.746 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-04-26T21:51:23.747 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-04-26T21:51:23.748 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2024-04-26T21:51:23.748 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53118.log.gz 2024-04-26T21:51:23.749 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-04-26T21:51:23.749 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-04-26T21:51:23.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40652.log 2024-04-26T21:51:23.750 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2024-04-26T21:51:23.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53358.log 2024-04-26T21:51:23.752 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-04-26T21:51:23.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-04-26T21:51:23.753 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40652.log.gz 2024-04-26T21:51:23.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46436.log 2024-04-26T21:51:23.754 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53358.log.gz 2024-04-26T21:51:23.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-04-26T21:51:23.756 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-04-26T21:51:23.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45094.log 2024-04-26T21:51:23.757 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46436.log.gz 2024-04-26T21:51:23.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40904.log 2024-04-26T21:51:23.758 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-04-26T21:51:23.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-26T21:51:23.759 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-04-26T21:51:23.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46180.log 2024-04-26T21:51:23.760 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40904.log.gz 2024-04-26T21:51:23.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-04-26T21:51:23.762 INFO:teuthology.orchestra.run.smithi106.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-26T21:51:23.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42287.log 2024-04-26T21:51:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46180.log: /var/log/ceph/ceph-client.admin.43111.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46180.log.gz 2024-04-26T21:51:23.764 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-04-26T21:51:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52187.log 2024-04-26T21:51:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-04-26T21:51:23.765 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42287.log.gz 2024-04-26T21:51:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52187.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T21:51:23.767 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52187.log.gz 2024-04-26T21:51:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46580.log 2024-04-26T21:51:23.768 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-04-26T21:51:23.769 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2024-04-26T21:51:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51622.log 2024-04-26T21:51:23.770 INFO:teuthology.orchestra.run.smithi106.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T21:51:23.771 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46580.log.gz 2024-04-26T21:51:23.771 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53256.log 2024-04-26T21:51:23.772 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2024-04-26T21:51:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51538.log 2024-04-26T21:51:23.773 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51622.log.gz 2024-04-26T21:51:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51790.log 2024-04-26T21:51:23.774 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53256.log.gz 2024-04-26T21:51:23.774 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51538.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-04-26T21:51:23.775 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51292.log 2024-04-26T21:51:23.776 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46222.log 2024-04-26T21:51:23.776 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51790.log.gz 2024-04-26T21:51:23.777 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51292.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T21:51:23.777 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51292.log.gz 2024-04-26T21:51:23.778 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39853.log 2024-04-26T21:51:23.778 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46222.log.gz 2024-04-26T21:51:23.779 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T21:51:23.780 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39853.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26655.log 2024-04-26T21:51:23.780 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39853.log.gz 2024-04-26T21:51:23.782 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44412.log 2024-04-26T21:51:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.tmp-client.admin.26655.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26655.log.gz 2024-04-26T21:51:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-04-26T21:51:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53034.log 2024-04-26T21:51:23.785 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44412.log.gz 2024-04-26T21:51:23.786 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2024-04-26T21:51:23.786 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-04-26T21:51:23.787 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-04-26T21:51:23.787 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-04-26T21:51:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46958.log 2024-04-26T21:51:23.789 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2024-04-26T21:51:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T21:51:23.790 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-04-26T21:51:23.791 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2024-04-26T21:51:23.791 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46958.log.gz 2024-04-26T21:51:23.792 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52482.log 2024-04-26T21:51:23.794 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50990.log 2024-04-26T21:51:23.794 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gz 2024-04-26T21:51:23.796 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52482.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52482.log.gz 2024-04-26T21:51:23.796 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-04-26T21:51:23.796 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50990.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50990.log.gz 2024-04-26T21:51:23.797 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52229.log 2024-04-26T21:51:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41731.log 2024-04-26T21:51:23.799 INFO:teuthology.orchestra.run.smithi106.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-04-26T21:51:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52229.log: gzip -5 --verbose -- 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52229.log.gz 2024-04-26T21:51:23.800 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.0.47428.log 2024-04-26T21:51:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41731.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38631.log 2024-04-26T21:51:23.801 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-26T21:51:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.47428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52020.log 2024-04-26T21:51:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38631.log.gz 2024-04-26T21:51:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52020.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2024-04-26T21:51:23.806 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-04-26T21:51:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53862.log 2024-04-26T21:51:23.809 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-04-26T21:51:23.809 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53862.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.47428.log.gz 2024-04-26T21:51:23.810 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53862.log.gz 2024-04-26T21:51:23.810 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45430.log 2024-04-26T21:51:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2024-04-26T21:51:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49957.log 2024-04-26T21:51:23.813 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45430.log.gz 2024-04-26T21:51:23.814 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51958.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2024-04-26T21:51:23.816 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49957.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49957.log.gz 2024-04-26T21:51:23.816 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49568.log 2024-04-26T21:51:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-04-26T21:51:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-04-26T21:51:23.819 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49568.log.gz 2024-04-26T21:51:23.820 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-04-26T21:51:23.821 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-26T21:51:23.822 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52992.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-04-26T21:51:23.822 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-04-26T21:51:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-04-26T21:51:23.824 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-04-26T21:51:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2024-04-26T21:51:23.825 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-04-26T21:51:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44128.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-04-26T21:51:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45178.log 2024-04-26T21:51:23.828 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2024-04-26T21:51:23.828 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40694.log 2024-04-26T21:51:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-04-26T21:51:23.831 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45178.log.gz 2024-04-26T21:51:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44043.log 2024-04-26T21:51:23.832 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40694.log.gz 2024-04-26T21:51:23.833 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45598.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-04-26T21:51:23.834 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-04-26T21:51:23.834 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44043.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44043.log.gz 2024-04-26T21:51:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-04-26T21:51:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43642.log 2024-04-26T21:51:23.837 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz 2024-04-26T21:51:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52796.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-04-26T21:51:23.839 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43642.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43642.log.gz 2024-04-26T21:51:23.839 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46538.log 2024-04-26T21:51:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-04-26T21:51:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46832.log 2024-04-26T21:51:23.843 INFO:teuthology.orchestra.run.smithi204.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T21:51:23.843 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2024-04-26T21:51:23.844 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42481.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-04-26T21:51:23.844 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2024-04-26T21:51:23.845 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-04-26T21:51:23.845 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46832.log.gz 2024-04-26T21:51:23.847 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38716.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2024-04-26T21:51:23.848 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45136.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-04-26T21:51:23.848 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2024-04-26T21:51:23.849 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40988.log 2024-04-26T21:51:23.851 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51664.log 2024-04-26T21:51:23.851 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-04-26T21:51:23.852 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40988.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz 2024-04-26T21:51:23.854 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51664.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51664.log.gz 2024-04-26T21:51:23.854 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T21:51:23.855 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-04-26T21:51:23.857 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42775.log 2024-04-26T21:51:23.858 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46496.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-04-26T21:51:23.859 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42775.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42775.log.gz 2024-04-26T21:51:23.860 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52754.log 2024-04-26T21:51:23.861 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52313.log 2024-04-26T21:51:23.862 INFO:teuthology.orchestra.run.smithi204.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T21:51:23.863 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52754.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52754.log.gz 2024-04-26T21:51:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39056.log 2024-04-26T21:51:23.865 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52313.log.gz 2024-04-26T21:51:23.866 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-04-26T21:51:23.868 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39056.log.gz 2024-04-26T21:51:23.868 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.44085.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2024-04-26T21:51:23.869 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2024-04-26T21:51:23.870 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43542.log 2024-04-26T21:51:23.871 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49999.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2024-04-26T21:51:23.872 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47042.log 2024-04-26T21:51:23.873 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43542.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43542.log.gz 2024-04-26T21:51:23.874 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39856.log 2024-04-26T21:51:23.875 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47042.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47042.log.gz 2024-04-26T21:51:23.876 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-04-26T21:51:23.876 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39856.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39856.log.gz 2024-04-26T21:51:23.877 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-04-26T21:51:23.879 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39522.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-04-26T21:51:23.879 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44968.log 2024-04-26T21:51:23.880 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40526.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-04-26T21:51:23.881 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44842.log 2024-04-26T21:51:23.882 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44968.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44968.log.gz 2024-04-26T21:51:23.883 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39859.log 2024-04-26T21:51:23.884 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44842.log.gz 2024-04-26T21:51:23.885 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-04-26T21:51:23.886 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39859.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39859.log.gz 2024-04-26T21:51:23.887 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41575.log 2024-04-26T21:51:23.887 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T21:51:23.888 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51748.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-04-26T21:51:23.890 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T21:51:23.891 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.41575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45262.log 2024-04-26T21:51:23.891 INFO:teuthology.orchestra.run.smithi106.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.41575.log.gz 2024-04-26T21:51:23.892 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48577.log 2024-04-26T21:51:23.893 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45262.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45262.log.gz 2024-04-26T21:51:23.894 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-04-26T21:51:23.895 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48577.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48577.log.gz 2024-04-26T21:51:23.896 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44884.log 2024-04-26T21:51:23.896 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39520.log: 91.8% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-04-26T21:51:23.897 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T21:51:23.897 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2024-04-26T21:51:23.898 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44884.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-04-26T21:51:23.899 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44884.log.gz 2024-04-26T21:51:23.900 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53213.log 2024-04-26T21:51:23.900 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2024-04-26T21:51:23.901 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43500.log: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T21:51:23.901 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-04-26T21:51:23.901 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-04-26T21:51:23.903 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53213.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53213.log.gz 2024-04-26T21:51:23.903 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-04-26T21:51:23.904 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51418.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-04-26T21:51:23.904 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-04-26T21:51:23.905 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40195.log 2024-04-26T21:51:23.906 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-04-26T21:51:23.907 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38589.log 2024-04-26T21:51:23.907 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-04-26T21:51:23.908 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39224.log 2024-04-26T21:51:23.908 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40195.log.gz 2024-04-26T21:51:23.910 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48010.log 2024-04-26T21:51:23.910 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38589.log.gz 2024-04-26T21:51:23.911 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53400.log 2024-04-26T21:51:23.911 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39224.log.gz 2024-04-26T21:51:23.912 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2024-04-26T21:51:23.912 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-04-26T21:51:23.914 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-04-26T21:51:23.914 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53400.log.gz 2024-04-26T21:51:23.914 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T21:51:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-04-26T21:51:23.915 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gz 2024-04-26T21:51:23.916 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53944.log 2024-04-26T21:51:23.916 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-04-26T21:51:23.917 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-04-26T21:51:23.917 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-04-26T21:51:23.918 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2024-04-26T21:51:23.919 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53944.log.gz 2024-04-26T21:51:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-04-26T21:51:23.920 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-04-26T21:51:23.921 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-04-26T21:51:23.921 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz 2024-04-26T21:51:23.922 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2024-04-26T21:51:23.922 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-04-26T21:51:23.924 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-04-26T21:51:23.924 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2024-04-26T21:51:23.925 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-04-26T21:51:23.925 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2024-04-26T21:51:23.926 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50041.log 2024-04-26T21:51:23.926 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-04-26T21:51:23.927 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-04-26T21:51:23.928 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-04-26T21:51:23.929 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46748.log 2024-04-26T21:51:23.929 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50041.log.gz 2024-04-26T21:51:23.930 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40200.log 2024-04-26T21:51:23.930 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-04-26T21:51:23.931 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-04-26T21:51:23.931 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46748.log.gz 2024-04-26T21:51:23.932 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46264.log 2024-04-26T21:51:23.933 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-04-26T21:51:23.934 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46096.log 2024-04-26T21:51:23.934 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-04-26T21:51:23.935 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40150.log 2024-04-26T21:51:23.935 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46264.log.gz 2024-04-26T21:51:23.936 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40736.log 2024-04-26T21:51:23.936 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46096.log.gz 2024-04-26T21:51:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-04-26T21:51:23.938 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40150.log.gz 2024-04-26T21:51:23.939 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40778.log 2024-04-26T21:51:23.939 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40736.log.gz 2024-04-26T21:51:23.940 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46790.log 2024-04-26T21:51:23.940 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz 2024-04-26T21:51:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-04-26T21:51:23.942 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40778.log.gz 2024-04-26T21:51:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2024-04-26T21:51:23.943 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46790.log.gz 2024-04-26T21:51:23.944 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48791.log 2024-04-26T21:51:23.944 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-04-26T21:51:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-04-26T21:51:23.945 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2024-04-26T21:51:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2024-04-26T21:51:23.947 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48791.log.gz 2024-04-26T21:51:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48273.log 2024-04-26T21:51:23.948 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-04-26T21:51:23.949 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-04-26T21:51:23.949 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2024-04-26T21:51:23.950 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-04-26T21:51:23.951 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48273.log.gz 2024-04-26T21:51:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46392.log 2024-04-26T21:51:23.952 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-04-26T21:51:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53076.log 2024-04-26T21:51:23.953 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-04-26T21:51:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-04-26T21:51:23.954 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46392.log.gz 2024-04-26T21:51:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-04-26T21:51:23.956 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53076.log.gz 2024-04-26T21:51:23.956 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T21:51:23.957 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39350.log 2024-04-26T21:51:23.957 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-04-26T21:51:23.958 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51150.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-04-26T21:51:23.958 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-04-26T21:51:23.960 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-04-26T21:51:23.960 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m0.347s 2024-04-26T21:51:23.960 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m1.444s 2024-04-26T21:51:23.960 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.225s 2024-04-26T21:51:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-04-26T21:51:23.961 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39350.log.gz 2024-04-26T21:51:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44454.log 2024-04-26T21:51:23.962 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-04-26T21:51:23.963 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-04-26T21:51:23.963 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-04-26T21:51:23.964 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53171.log 2024-04-26T21:51:23.964 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44454.log.gz 2024-04-26T21:51:23.965 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51916.log 2024-04-26T21:51:23.965 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-04-26T21:51:23.966 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40190.log 2024-04-26T21:51:23.966 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53171.log.gz 2024-04-26T21:51:23.967 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-04-26T21:51:23.968 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51916.log.gz 2024-04-26T21:51:23.969 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-04-26T21:51:23.969 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40190.log.gz 2024-04-26T21:51:23.970 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39266.log 2024-04-26T21:51:23.970 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-04-26T21:51:23.971 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-04-26T21:51:23.971 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-04-26T21:51:23.972 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-04-26T21:51:23.973 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39266.log.gz 2024-04-26T21:51:23.974 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47000.log 2024-04-26T21:51:23.974 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-04-26T21:51:23.975 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51874.log 2024-04-26T21:51:23.975 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-04-26T21:51:23.976 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49351.log 2024-04-26T21:51:23.976 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47000.log.gz 2024-04-26T21:51:23.978 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44800.log 2024-04-26T21:51:23.978 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51874.log.gz 2024-04-26T21:51:23.979 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-04-26T21:51:23.979 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49351.log.gz 2024-04-26T21:51:23.980 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-04-26T21:51:23.980 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44800.log.gz 2024-04-26T21:51:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-04-26T21:51:23.982 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-04-26T21:51:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52567.log 2024-04-26T21:51:23.983 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-04-26T21:51:23.984 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39531.log 2024-04-26T21:51:23.984 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-04-26T21:51:23.985 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53652.log 2024-04-26T21:51:23.985 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52567.log.gz 2024-04-26T21:51:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2024-04-26T21:51:23.987 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39531.log.gz 2024-04-26T21:51:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-04-26T21:51:23.988 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53652.log.gz 2024-04-26T21:51:23.989 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48619.log 2024-04-26T21:51:23.989 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz 2024-04-26T21:51:23.990 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48847.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-04-26T21:51:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2024-04-26T21:51:23.992 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48619.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48619.log.gz 2024-04-26T21:51:23.992 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-26T21:51:23.993 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46138.log 2024-04-26T21:51:23.993 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2024-04-26T21:51:23.995 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2024-04-26T21:51:23.995 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-04-26T21:51:23.996 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53526.log 2024-04-26T21:51:23.996 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46138.log.gz 2024-04-26T21:51:23.997 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40202.log 2024-04-26T21:51:23.997 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2024-04-26T21:51:23.999 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2024-04-26T21:51:23.999 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53526.log.gz 2024-04-26T21:51:24.000 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43600.log 2024-04-26T21:51:24.000 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40202.log.gz 2024-04-26T21:51:24.001 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-04-26T21:51:24.001 INFO:teuthology.orchestra.run.smithi106.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2024-04-26T21:51:24.003 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50381.log 2024-04-26T21:51:24.003 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43600.log.gz 2024-04-26T21:51:24.004 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46348.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-04-26T21:51:24.004 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2024-04-26T21:51:24.005 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50381.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T21:51:24.006 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50381.log.gz 2024-04-26T21:51:24.007 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-04-26T21:51:24.007 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2024-04-26T21:51:24.008 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48147.log 2024-04-26T21:51:24.009 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-04-26T21:51:24.009 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-04-26T21:51:24.010 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48147.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48147.log.gz 2024-04-26T21:51:24.011 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47651.log 2024-04-26T21:51:24.012 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz -- 2024-04-26T21:51:24.012 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.46706.log 2024-04-26T21:51:24.014 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46306.log 2024-04-26T21:51:24.014 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47651.log.gz 2024-04-26T21:51:24.015 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46706.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46706.log.gz 2024-04-26T21:51:24.016 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.46306.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39529.log 2024-04-26T21:51:24.016 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46306.log.gz 2024-04-26T21:51:24.017 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50864.log 2024-04-26T21:51:24.019 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49818.log 2024-04-26T21:51:24.019 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39529.log.gz 2024-04-26T21:51:24.021 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50864.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50864.log.gz 2024-04-26T21:51:24.021 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49818.log: /var/log/ceph/ceph-client.admin.45724.log 2024-04-26T21:51:24.021 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49818.log.gz 2024-04-26T21:51:24.022 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-04-26T21:51:24.024 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39862.log 2024-04-26T21:51:24.024 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-04-26T21:51:24.025 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46622.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-04-26T21:51:24.026 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-04-26T21:51:24.026 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39862.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39862.log.gz 2024-04-26T21:51:24.027 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51048.log 2024-04-26T21:51:24.029 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52669.log 2024-04-26T21:51:24.029 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-04-26T21:51:24.030 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51048.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51048.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-04-26T21:51:24.030 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-26T21:51:24.032 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2024-04-26T21:51:24.032 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52669.log.gz 2024-04-26T21:51:24.033 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45010.log 2024-04-26T21:51:24.033 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-04-26T21:51:24.034 INFO:teuthology.orchestra.run.smithi106.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T21:51:24.034 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-04-26T21:51:24.035 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2024-04-26T21:51:24.035 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47084.log 2024-04-26T21:51:24.036 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45010.log.gz 2024-04-26T21:51:24.037 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40862.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz -5 2024-04-26T21:51:24.037 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-04-26T21:51:24.037 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45220.log 2024-04-26T21:51:24.039 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43684.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47084.log.gz 2024-04-26T21:51:24.039 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-04-26T21:51:24.039 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-04-26T21:51:24.040 INFO:teuthology.orchestra.run.smithi106.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T21:51:24.040 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44926.log 2024-04-26T21:51:24.040 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45220.log.gz 2024-04-26T21:51:24.040 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T21:51:24.041 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2024-04-26T21:51:24.042 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-04-26T21:51:24.042 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-04-26T21:51:24.042 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44926.log.gz 2024-04-26T21:51:24.043 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49393.log 2024-04-26T21:51:24.043 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2024-04-26T21:51:24.044 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50255.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-04-26T21:51:24.044 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-04-26T21:51:24.045 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43237.log 2024-04-26T21:51:24.045 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49393.log.gz 2024-04-26T21:51:24.046 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-04-26T21:51:24.047 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-04-26T21:51:24.047 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40203.log 2024-04-26T21:51:24.048 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43237.log.gz 2024-04-26T21:51:24.049 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50087.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-04-26T21:51:24.049 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45514.log 2024-04-26T21:51:24.049 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-04-26T21:51:24.050 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40203.log.gz 2024-04-26T21:51:24.051 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52525.log 2024-04-26T21:51:24.051 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45514.log.gz 2024-04-26T21:51:24.052 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38844.log 2024-04-26T21:51:24.052 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz 2024-04-26T21:51:24.053 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53820.log 2024-04-26T21:51:24.053 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52525.log.gz 2024-04-26T21:51:24.054 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.38844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51234.log 2024-04-26T21:51:24.054 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38844.log.gz 2024-04-26T21:51:24.055 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53694.log 2024-04-26T21:51:24.055 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53820.log.gz 2024-04-26T21:51:24.056 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-04-26T21:51:24.057 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51234.log.gz 2024-04-26T21:51:24.057 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2024-04-26T21:51:24.057 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53694.log.gz 2024-04-26T21:51:24.058 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2024-04-26T21:51:24.059 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-04-26T21:51:24.059 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39014.log 2024-04-26T21:51:24.060 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2024-04-26T21:51:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40610.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2024-04-26T21:51:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.39014.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39014.log.gz 2024-04-26T21:51:24.072 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T21:51:24.221 INFO:teuthology.orchestra.run.smithi106.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T21:51:24.224 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-26T21:51:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m0.614s 2024-04-26T21:51:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m1.550s 2024-04-26T21:51:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m1.027s 2024-04-26T21:51:24.225 INFO:tasks.ceph:Archiving logs... 2024-04-26T21:51:24.226 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi106/log 2024-04-26T21:51:24.227 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:51:24.738 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi204/log 2024-04-26T21:51:24.738 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:51:25.020 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:51:25.031 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:51:25.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:51:25.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:51:25.178 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:51:25.188 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:51:25.188 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:51:25.196 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:51:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:51:25.252 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:51:25.252 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:51:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-26T21:51:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +1270us[+1282us] +/- 52ms 2024-04-26T21:51:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -620us[ -620us] +/- 56ms 2024-04-26T21:51:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -1576us[-1576us] +/- 72ms 2024-04-26T21:51:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:51:25.263 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:51:25.263 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-26T21:51:25.263 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1290us[+1319us] +/- 52ms 2024-04-26T21:51:25.263 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -554us[ -525us] +/- 57ms 2024-04-26T21:51:25.263 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -1571us[-1571us] +/- 72ms 2024-04-26T21:51:25.264 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:51:25.264 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:51:25.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:51:25.275 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:51:25.293 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:51:25.494 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:51:25.682 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:51:25.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:25.774 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:51:25.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:25.865 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:51:25.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:51:25.912 INFO:teuthology.task.internal:Duration was 886.005162 seconds 2024-04-26T21:51:25.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:51:25.943 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:51:25.943 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:51:25.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:51:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:51:26.096 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:51:26.407 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:51:26.407 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-26T21:51:26.408 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:51:26.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:51:26.464 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:51:26.541 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:51:26.542 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:51:26.550 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:51:26.631 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:51:26.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:51:26.637 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:51:26.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:51:26.818 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:51:26.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:51:26.899 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:51:26.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:51:26.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:27.004 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:27.096 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-26T21:51:27.103 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-04-26T21:51:27.145 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:27.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:27.193 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:27.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:27.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:51:27.259 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:51:27.260 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi106 2024-04-26T21:51:27.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:51:27.357 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230/remote/smithi204 2024-04-26T21:51:27.357 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:51:27.454 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:51:27.454 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:51:27.462 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:51:27.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:51:27.532 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:51:27.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:51:27.588 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:51:27.588 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:51:27.596 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:51:27.645 INFO:teuthology.orchestra.run.smithi106.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:51 /home/ubuntu/cephtest 2024-04-26T21:51:27.653 INFO:teuthology.orchestra.run.smithi204.stdout: 393242 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:51 /home/ubuntu/cephtest 2024-04-26T21:51:27.656 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:51:27.667 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:51:27.787 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T21:51:27.812 DEBUG:teuthology.nuke:shortname: smithi106 2024-04-26T21:51:27.812 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:51:27.834 DEBUG:teuthology.nuke:shortname: smithi204 2024-04-26T21:51:27.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:51:27.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:27:04.591834', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGvZg1Zby3QB9t81i0ElWC414PbVAsNiey83rUHUZCko6B8eS6pLsKEIMacb1QCNwzf/k21snr7aND/AUTJfSgk='} 2024-04-26T21:51:27.857 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:27:04.590358', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvsDwOjxTbut750N71/3tg1jCt0r43pR8LS3tiL6W+phHCPmd1t7l2VKVcjeP7Oe+yPoy4+oW+C+So7jUXfp1M='} 2024-04-26T21:51:27.882 INFO:teuthology.orchestra.console.smithi106:Power off 2024-04-26T21:51:27.883 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T21:51:27.907 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-04-26T21:51:27.907 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:27.923 INFO:teuthology.orchestra.console.smithi204:Power off 2024-04-26T21:51:27.924 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T21:51:27.937 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-26T21:51:27.981 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-04-26T21:51:27.982 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:28.006 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-26T21:51:31.939 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:32.007 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:32.050 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-26T21:51:32.116 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-26T21:51:36.055 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:36.117 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:36.166 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-04-26T21:51:36.227 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-26T21:51:36.267 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-04-26T21:51:40.230 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:40.340 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-04-26T21:51:40.441 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-04-26T21:51:40.546 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 886.0051617622375 failure_reason: 'Test failure: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc)' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b2846c68420d4b9790e62b2bc4be54ca status: fail success: false 2024-04-26T21:51:40.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:51:40.628 INFO:teuthology.run:FAIL